-- Logs begin at Tue 2022-09-13 19:06:49 UTC, end at Tue 2022-09-13 20:41:47 UTC. -- Sep 13 19:35:55.177221 dv-cinder-828807-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 19:35:55.185148 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 19:35:55.188313 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 19:35:55 2022] *** Sep 13 19:35:55.188313 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 19:35:55.188313 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 19:35:55.188313 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: nodename: dv-cinder-828807-6-iscsi Sep 13 19:35:55.188313 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: machine: x86_64 Sep 13 19:35:55.188313 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: clock source: unix Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: pcre jit disabled Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: detected number of CPU cores: 4 Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: current working directory: / Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: detected binary path: /usr/bin/uwsgi-core Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: your processes number limit is 47680 Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: your memory page size is 4096 bytes Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: detected max file descriptor number: 2048 Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: lock engine: pthread robust mutexes Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: thunder lock: enabled Sep 13 19:35:55.188714 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 19:35:55.189304 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 19:35:55.229529 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: Python main interpreter initialized at 0x5612bd5ad9c0 Sep 13 19:35:55.229529 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: python threads support enabled Sep 13 19:35:55.229529 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: your server socket listen backlog is limited to 100 connections Sep 13 19:35:55.229529 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: your mercy for graceful operations on workers is 90 seconds Sep 13 19:35:55.229955 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 19:35:55.230038 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: *** Operational MODE: preforking *** Sep 13 19:35:55.230273 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: *** uWSGI is running in multiple interpreter mode *** Sep 13 19:35:55.230273 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: spawned uWSGI master process (pid: 93104) Sep 13 19:35:55.230490 dv-cinder-828807-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 19:35:55.231576 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: spawned uWSGI worker 1 (pid: 93112, cores: 1) Sep 13 19:35:55.231576 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: spawned uWSGI worker 2 (pid: 93113, cores: 1) Sep 13 19:35:55.231576 dv-cinder-828807-6-iscsi devstack@keystone.service[93104]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 19:35:59.500146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:59.500146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: from cryptography.utils import int_from_bytes Sep 13 19:35:59.501147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:59.501147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: from cryptography.utils import int_from_bytes Sep 13 19:35:59.502091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:59.502091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: from cryptography.utils import int_from_bytes Sep 13 19:35:59.502548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 19:35:59.502548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: from cryptography.utils import int_from_bytes Sep 13 19:35:59.551659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:35:59.551858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:35:59.552022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] command line args: [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:35:59.552290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:35:59.552467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ================================================================================ {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:35:59.552720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] admin_token = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.552965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] config_dir = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.553153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.553334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] config_source = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.553514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] debug = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.553713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.553985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 19:35:59.553985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 19:35:59.554116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.554180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] command line args: [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 19:35:59.554329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.554430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 19:35:59.554534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.554637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ================================================================================ {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 19:35:59.554739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.554848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] admin_token = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.554944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] config_dir = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_config_append = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] config_source = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_dir = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] debug = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_file = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.555883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] insecure_debug = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.556850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_config_append = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_dir = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_file = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.557907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.558998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] publish_errors = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.559988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.560894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] notification_format = cadf {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] standard_threads = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] publish_errors = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.561937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] use_journal = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] use_json = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] use_stderr = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] use_syslog = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.562903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] standard_threads = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.562979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.563049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] strict_password_check = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.563157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.563271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.563395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.563473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] use_eventlog = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.563611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] use_journal = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.563692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.563789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] use_json = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.563867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.563965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] use_stderr = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.564097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.564175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] use_syslog = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.564333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.564415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] watch_log_file = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 19:35:59.564579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.564668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.564824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.564901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.565902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.566884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.567986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.568929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.569998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.570938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.571876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.572935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.573950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.574914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.575971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.576896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.577941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.578882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.579968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.580910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.581962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.enabled = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.582900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.583857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.584931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.585956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.586867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.587946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.588025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.external = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.588172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.588251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.588406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.588492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.password = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] auth.token = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.589906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.external = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.mapped = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.590938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.password = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] auth.token = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.591876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] catalog.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.592976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.593957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.594928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.595972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.596930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.597912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.598874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.599927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.600974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.601914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.602864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.603957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.604037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.604195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.604275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.604985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.605958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.606983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.607921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.608900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.password = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.609984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.610880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.611980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.612923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.613880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.614951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.615881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.616973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.617953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.618900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] policy.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.619908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] receipt.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.620901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.621894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] role.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] role.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.622988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] revoke.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.623955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] role.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] role.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] role.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] role.list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.624979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.625981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626624 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.626876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.627992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.628957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.629962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.630910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.631858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.632963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.provider = fernet {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.633936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.634991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] trust.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.connection = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.635964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.636927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.connection = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.637903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.638984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.639836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93112) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:35:59.639916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.640149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.640397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.640634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.640876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.641106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.641346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.641581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 19:35:59.641766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=93113) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 19:35:59.669663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.669921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.670151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.670745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.670981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.671210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.678484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.679546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.706669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.706896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.707665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.707900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.760373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.760501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.809333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.809560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.809648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.809822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.966509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.966740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.966831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.966955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.967855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.990669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.990842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.990950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.991946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.992036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:35:59.992159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.003040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.003322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.016937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93112) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.017828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=93113) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 19:36:00.034532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.034742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:36:00.034825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.035033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:36:00.035115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.035265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:36:00.035378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.035538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.035619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 19:36:00.035802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.035892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.036064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.036146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.036322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.036402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.036587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.045939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.046219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.046313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.046491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.046574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.046775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.046859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.047061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.053691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.054054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.056604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.056887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.056975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.057172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.057255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.057456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.058879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.059959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.060147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.082898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.083333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.084359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.084644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.084811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.085098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.089329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.089617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.089838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.089922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.090124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.090409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.096200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.096484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.096740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.097030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.102047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.102349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.102642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.102724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.102931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.103852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.104041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.104127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.104314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.104613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.119764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.120559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.120999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.121280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.124368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.125164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.127568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.127866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.128437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.128742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.132005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.132972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.135185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.136796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.138947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.139251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.139536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.139818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.140075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.140157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.140379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.140461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.140670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.140957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.141236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.141534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.156086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.156372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.156638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.156900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.157367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.157659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.157956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.158225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.163741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.165096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.165289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.166677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.166845 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.167157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.168239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.168527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.170001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.170287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.170570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.170865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.171210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.171538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.171622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.171764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.172059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.172455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.182337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.183152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.183436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.183716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.184098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.184192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.184397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.184479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.184677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.184797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.184984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.185368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.185659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.185977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.200531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.201935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.203452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.204873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.206371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.207805 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.207934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.209246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.209328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.209545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.209967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.210769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.211054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.211450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.217147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.218082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.219568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.219850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.222779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.223345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.223807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.223890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.224228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.224362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.224758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.225041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.233268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.233557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.233930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.234015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.234301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.234586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.241032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93112) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.241317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.241600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.241682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=93113) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 19:36:00.241904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.241987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.242183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.242273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.242458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.242541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.242734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.242816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.243943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93112) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.244933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.245805 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=93113) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 19:36:00.279470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5612bd5ad9c0 pid: 93112 (default app) Sep 13 19:36:00.280260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5612bd5ad9c0 pid: 93113 (default app) Sep 13 19:36:00.282705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b7e83-de91-441f-8b26-52cf0c854017 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:00.282993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b7e83-de91-441f-8b26-52cf0c854017 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:00.283263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b7e83-de91-441f-8b26-52cf0c854017 None None] PATH_INFO: `/v3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:00.287430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Tue Sep 13 19:36:00 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:07.022140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.022655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.022761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.026956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:07.032405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:07.032836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:07.032985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:07.103694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: INFO dbcounter [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] Registered counter for database keystone Sep 13 19:36:07.123206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG oslo_db.sqlalchemy.engines [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93112) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:36:07.129283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writer thread running {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:36:07.306987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG passlib.handlers.bcrypt [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93112) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:36:07.307380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG passlib.handlers.bcrypt [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93112) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:36:07.366778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:07.368341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:07.503158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 13 19:36:07.952960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.023326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: INFO dbcounter [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] Registered counter for database keystone Sep 13 19:36:08.045118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG oslo_db.sqlalchemy.engines [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=93113) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 19:36:08.051398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writer thread running {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 19:36:08.169016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310d900-5b60-4693-a13d-1bcfa58332d3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.173067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.173687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.173990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.174262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.182385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.233813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:36:08.234197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2daa3c34-4634-44e2-a2c3-a93b47e26edb admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.243645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Sep 13 19:36:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:36:08.249606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:08.266949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bdc044-864b-4403-a6e8-6f10da9eea84 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:08.271378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:08.272491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:08.272776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:08.273049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] PATH_INFO: `/v3/projects/35cfa2e060644668a62b0b52ed4b9871` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:08.274672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] RBAC: Authorizing `identity:get_project(project_id=35cfa2e060644668a62b0b52ed4b9871)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:08.328429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 19:36:08.328773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c633e3-a7f0-4141-9805-e86538b9c8db admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:08.333298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Tue Sep 13 19:36:08 2022] GET /identity/v3/projects/35cfa2e060644668a62b0b52ed4b9871 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:10.428772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d50bff-9cbd-4696-bc6f-3407610366ea None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.429309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d50bff-9cbd-4696-bc6f-3407610366ea None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.429309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d50bff-9cbd-4696-bc6f-3407610366ea None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.433387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:10.438771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.439145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.439320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.460147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:10.461654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:10.470112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 19:36:10.735720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d50bff-9cbd-4696-bc6f-3407610366ea None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.748341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d50bff-9cbd-4696-bc6f-3407610366ea None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.752458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82db9bb3-dbd9-4859-998a-ca7509263a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.753049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9bb3-dbd9-4859-998a-ca7509263a41 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.753323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9bb3-dbd9-4859-998a-ca7509263a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.753593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82db9bb3-dbd9-4859-998a-ca7509263a41 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.761080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db9bb3-dbd9-4859-998a-ca7509263a41 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.763462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82db9bb3-dbd9-4859-998a-ca7509263a41 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.772029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:36:10.777269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:10.786366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eb2d5e-2902-4d13-98c3-38715e5b57b6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:10.790671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3655588-3d1e-4a24-b80d-5dcf9d9d1e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:10.791287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3655588-3d1e-4a24-b80d-5dcf9d9d1e1e admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:10.791563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3655588-3d1e-4a24-b80d-5dcf9d9d1e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:10.791837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3655588-3d1e-4a24-b80d-5dcf9d9d1e1e admin admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:10.793135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3655588-3d1e-4a24-b80d-5dcf9d9d1e1e admin admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:10.794975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3655588-3d1e-4a24-b80d-5dcf9d9d1e1e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:10.796485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Tue Sep 13 19:36:10 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:13.002500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaf1a3-e1c0-4035-8608-44b12125475a None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.003027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaf1a3-e1c0-4035-8608-44b12125475a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.003027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86aaf1a3-e1c0-4035-8608-44b12125475a None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.004353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:13.009629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.010065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.010215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.027811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:13.029501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:13.070199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967623f-694f-4cac-9f98-17e2df731f0e None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.070481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967623f-694f-4cac-9f98-17e2df731f0e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.070755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967623f-694f-4cac-9f98-17e2df731f0e None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.072264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:13.077211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.077489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.077790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.112244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:13.120261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG passlib.handlers.bcrypt [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=93113) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 19:36:13.120667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG passlib.handlers.bcrypt [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=93113) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 19:36:13.170183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:13.171739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:13.269412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:13.382557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.407060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.485893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f969b-ff42-43cc-a97b-91f005c46b96 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.518664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.519292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.519571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.519847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.531138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:13.539893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:13.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:13.541843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:13.542470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.542547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:13.546582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.626832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.642511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191245d-c6c0-40e4-8bae-a303ba42740f None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.644748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.647395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377884cd-ef58-4682-a54f-b2685c04e6d3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.679112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.680081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.680431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.680774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.681397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.682024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.682304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.682574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.683232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:13.683532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.685425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.696905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:13.703140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.747131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.749167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.799471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b1b44-396c-4e1b-bc25-65064a06bcdc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.833751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.834397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.834678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.834959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.836121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.837584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.839273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 9/16] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:13.844655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.848635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 8/17] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:13.854107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:13.856863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.859607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:13.945714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caab690a-6021-45a9-a8d2-1bb9e7da2975 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.953529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f15cef-0662-4394-909f-491001a9a195 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:13.980100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.980728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.981007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.981278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.982910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:13.984331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:13.985874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:13.986596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:13.987439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:13.987712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:13.987994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:13.991566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.007288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.008776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:14.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:14.027731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:14.028752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:14.028752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:14.028752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:14.028752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:14.033625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:14.070411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.089185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28029ae3-347b-4f83-a86c-930347b73192 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.089332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:14.124174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.124786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.125065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.125338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.126085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.domain.id=default, user.id=8e9932faac4043979f160bcfef7916ea) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.126388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.127816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bdfda3-0933-4514-91e7-0628093158d6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.134325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Sep 13 19:36:13 2022] GET /identity/v3/role_assignments?user.id=8e9932faac4043979f160bcfef7916ea&scope.domain.id=default&role.id=40a9647e8ec745c999c2457d053557a7 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:14.134739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae42bc-8ffe-4f45-a040-cf28e6dcccef None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:14.170657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:14.171279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:14.171556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:14.171828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:14.172489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:14.172789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:14.174241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3a9fd-aeb8-4f81-8107-fd3c5859f121 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:14.194138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:16.453549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b3e7b-6529-4b7b-95e2-3737242609e6 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.454080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b3e7b-6529-4b7b-95e2-3737242609e6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.454152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b3e7b-6529-4b7b-95e2-3737242609e6 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.455712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 12/22] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:16.461192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec71ec-ccf9-49f2-96fc-927d7bbc2b79 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.462155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec71ec-ccf9-49f2-96fc-927d7bbc2b79 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.462389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec71ec-ccf9-49f2-96fc-927d7bbc2b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.488629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b7ec71ec-ccf9-49f2-96fc-927d7bbc2b79 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:16.489862 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b7ec71ec-ccf9-49f2-96fc-927d7bbc2b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:16.587000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 11/23] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:16.868551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0831b1-a182-4360-a5fd-e89727736ab2 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.868898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0831b1-a182-4360-a5fd-e89727736ab2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.869098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0831b1-a182-4360-a5fd-e89727736ab2 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.874238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 13/24] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:16.879960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46940c9-7b60-497c-997b-2ff9eb93512e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:16.880562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46940c9-7b60-497c-997b-2ff9eb93512e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:16.881039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46940c9-7b60-497c-997b-2ff9eb93512e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:16.906695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c46940c9-7b60-497c-997b-2ff9eb93512e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:16.908212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c46940c9-7b60-497c-997b-2ff9eb93512e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:16.964928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:16.998506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 12/25] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:17.032619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f364805-e43b-416b-8e3d-23c42f4e5c18 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.032915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f364805-e43b-416b-8e3d-23c42f4e5c18 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.033167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f364805-e43b-416b-8e3d-23c42f4e5c18 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.034514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 14/26] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:17.047197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.047758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.048232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.074164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:17.074461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f364805-e43b-416b-8e3d-23c42f4e5c18 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.075868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:17.194225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 13/27] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:17.279586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f364805-e43b-416b-8e3d-23c42f4e5c18 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.321040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.321676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.321974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.322285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.333047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:17.338890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:17.339745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:17.340489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 15/28] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:17.346435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.347775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.357057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.531611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551f4b-8b75-4ced-a34c-d27d977a1f91 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.532270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38fa0c-8b62-42ef-a866-8be25f14e047 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.554116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.595576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.596196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.596470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.597081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.598259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.598883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.599153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.599468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.600705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:17.601006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.602406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.623377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.624904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.625417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 14/29] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:17.626473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:36:17.650214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:17.651111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:36:17.651899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:36:17.651899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:36:17.651899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:17.651899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 16/30] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:17.652853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.722444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.772082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81994fba-8337-4f66-89ba-8100df778038 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.776451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b29275-5fe8-486a-bec1-fabb7fa180f0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.814052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39471f7e-1a67-4802-bbbe-b7273c9af47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.814529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39471f7e-1a67-4802-bbbe-b7273c9af47d None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.814806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39471f7e-1a67-4802-bbbe-b7273c9af47d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.815075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39471f7e-1a67-4802-bbbe-b7273c9af47d None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.815690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.815821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39471f7e-1a67-4802-bbbe-b7273c9af47d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.816469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.816869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.817113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39471f7e-1a67-4802-bbbe-b7273c9af47d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.817270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.819168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.820693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.822876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 17/31] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:17.823720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.934820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cb82c0-c128-4159-8f94-ee518363fa6a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:17.954895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 15/32] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/roles => generated 225 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:17.956679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b125-43dd-4c57-97d8-4598209efed2 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.957164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b125-43dd-4c57-97d8-4598209efed2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.957636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b125-43dd-4c57-97d8-4598209efed2 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.959772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 16/33] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:17.960882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b125-43dd-4c57-97d8-4598209efed2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:17.961336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.974980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:17.975615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:17.975896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:17.976172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:17.977273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:17.977273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:17.978796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92d875a-cdfd-4ff2-af25-5d13322dc8a8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:17.997258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:17.999487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 18/34] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:18.000719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.000993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.001307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.022781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.024018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:18.101670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 19/35] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:18.102373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.138985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b125-43dd-4c57-97d8-4598209efed2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.211037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27845005-f042-49f6-bbe7-d0f2a0f3f168 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.225690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.226541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.226826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.227098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.227243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.230062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.231835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.233710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 17/36] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:18.239993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.262893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff35e1-6e18-48eb-8579-e49578da78c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.263815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff35e1-6e18-48eb-8579-e49578da78c8 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.265240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff35e1-6e18-48eb-8579-e49578da78c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.265698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff35e1-6e18-48eb-8579-e49578da78c8 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.267091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff35e1-6e18-48eb-8579-e49578da78c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.270208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff35e1-6e18-48eb-8579-e49578da78c8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.384209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:18.389083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5beeb91-47d7-4834-b25f-5b06966ac458 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.430358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.431431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.431901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.432361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] PATH_INFO: `/v3/domains/default/users/8e9932faac4043979f160bcfef7916ea/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.434875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8e9932faac4043979f160bcfef7916ea, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.436514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.459589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 18/37] 10.222.0.15 () {58 vars in 1361 bytes} [Tue Sep 13 19:36:18 2022] PUT /identity/v3/domains/default/users/8e9932faac4043979f160bcfef7916ea/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:18.460810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:18.483367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 20/38] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/projects => generated 301 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:18.534579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abda7b0-1657-48d5-864c-c876217dc1b0 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.535227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abda7b0-1657-48d5-864c-c876217dc1b0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.535227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abda7b0-1657-48d5-864c-c876217dc1b0 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.537246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 21/39] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:18.545299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fc00a-768b-4915-b8bb-1992ba3d9da2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.545704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fc00a-768b-4915-b8bb-1992ba3d9da2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.546014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fc00a-768b-4915-b8bb-1992ba3d9da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.582631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-743fc00a-768b-4915-b8bb-1992ba3d9da2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.584220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-743fc00a-768b-4915-b8bb-1992ba3d9da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:18.593388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a4d3b9-6a9a-4bbf-8912-1c554ca0972d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:18.642604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55899f5f-7cf2-40d7-b05a-c024255fd103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:18.643675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55899f5f-7cf2-40d7-b05a-c024255fd103 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.644163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55899f5f-7cf2-40d7-b05a-c024255fd103 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.644631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55899f5f-7cf2-40d7-b05a-c024255fd103 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.645590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55899f5f-7cf2-40d7-b05a-c024255fd103 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:18.647412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55899f5f-7cf2-40d7-b05a-c024255fd103 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:18.673653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 19/40] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/roles => generated 219 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:18.715512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 22/41] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:18.840414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aff50d-31fa-40c7-99a0-c4ce8f6928a3 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.840952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aff50d-31fa-40c7-99a0-c4ce8f6928a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.841451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7aff50d-31fa-40c7-99a0-c4ce8f6928a3 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.843115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 20/42] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:18.849906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:18.850508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:18.851018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:18.978398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:18.979969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:19.004850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aff50d-31fa-40c7-99a0-c4ce8f6928a3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.024912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.057770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 23/43] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:19.118044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7aff50d-31fa-40c7-99a0-c4ce8f6928a3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.151273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.151884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.152165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.152442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.153122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.154523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.171097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 21/44] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 19:36:19 2022] POST /identity/v3/roles => generated 223 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:19.322501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.350668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.429354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21ac6d-7553-48ea-91fa-954d7a36c251 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.462932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ca3a6-4d50-4f14-8d64-4660f5c36252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.463558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ca3a6-4d50-4f14-8d64-4660f5c36252 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.463838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ca3a6-4d50-4f14-8d64-4660f5c36252 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.464111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ca3a6-4d50-4f14-8d64-4660f5c36252 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.465203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ca3a6-4d50-4f14-8d64-4660f5c36252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.466809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ca3a6-4d50-4f14-8d64-4660f5c36252 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.468376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 24/45] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:19.474024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:19.550878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:19.574072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8e9b90-80c0-4889-97c0-ff5e5f75cd9a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:19.605579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740563d-f3f1-447b-93b3-2289f13593fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:19.606228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740563d-f3f1-447b-93b3-2289f13593fe None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:19.606502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740563d-f3f1-447b-93b3-2289f13593fe None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:19.607023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740563d-f3f1-447b-93b3-2289f13593fe None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:19.607671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740563d-f3f1-447b-93b3-2289f13593fe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:19.609100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1740563d-f3f1-447b-93b3-2289f13593fe None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:19.637050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 22/46] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:19 2022] POST /identity/v3/projects => generated 312 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:20.770343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baabf4-ea41-411e-b8ec-a11d0c40c829 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.770883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baabf4-ea41-411e-b8ec-a11d0c40c829 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.770883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0baabf4-ea41-411e-b8ec-a11d0c40c829 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.772401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 25/47] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:20.777338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:20.777714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:20.778130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:20.796078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:20.797553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:20.865147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 23/48] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:21.136880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baabf4-ea41-411e-b8ec-a11d0c40c829 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.228179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.236942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0baabf4-ea41-411e-b8ec-a11d0c40c829 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.270069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.270705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.270988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.271263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.281149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:21.285989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:21.286815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:21.287547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 26/49] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:21.292845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.349209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.394034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437959b4-11b9-4656-bf53-e6259171bd29 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.427720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.428332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.428614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.428887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.429552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:21.429891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.431268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.439827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 24/50] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.446635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.543795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61135464-d099-4fa3-90ad-7dc7438019bd None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.576857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb69415-af1c-4543-9207-c21b1fc8284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.577512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb69415-af1c-4543-9207-c21b1fc8284e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.577830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb69415-af1c-4543-9207-c21b1fc8284e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.578111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb69415-af1c-4543-9207-c21b1fc8284e None admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.579234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb69415-af1c-4543-9207-c21b1fc8284e None admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.580680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb69415-af1c-4543-9207-c21b1fc8284e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.582310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 27/51] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.587696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.776077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f854-6929-48fb-a3a2-88d376ac5128 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.776593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f854-6929-48fb-a3a2-88d376ac5128 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.776675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391f854-6929-48fb-a3a2-88d376ac5128 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.777979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 28/52] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:21.783206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.783486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.783759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.791117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f99ebf-53d5-4384-aa52-381c54520042 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.801837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:21.803031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:21.811652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.819123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:21.827731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.828369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.828654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.828928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.830095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.831550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:21.833127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 25/53] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:21.838801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:21.870282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 29/54] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:21.956697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea57e86-8873-44fa-b1de-26bc4ee839e1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:21.988932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:21.989528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:21.989845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:21.990120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:21.990839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.domain.id=default, user.id=8e9932faac4043979f160bcfef7916ea) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:21.991145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:21.992566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.000847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 26/55] 10.222.0.15 () {56 vars in 1322 bytes} [Tue Sep 13 19:36:21 2022] GET /identity/v3/role_assignments?user.id=8e9932faac4043979f160bcfef7916ea&scope.domain.id=default&role.id=40a9647e8ec745c999c2457d053557a7 => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:22.141880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.268816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef790fa-4b62-49c5-b26a-4ce4681ef01c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.300859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a9f33-78b3-406e-84d8-8172fed3fbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.301479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a9f33-78b3-406e-84d8-8172fed3fbf2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.301782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a9f33-78b3-406e-84d8-8172fed3fbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.302058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a9f33-78b3-406e-84d8-8172fed3fbf2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.303138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a9f33-78b3-406e-84d8-8172fed3fbf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.304572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a9f33-78b3-406e-84d8-8172fed3fbf2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.306156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 30/56] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:22.312279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:22.363014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:22.408070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7f0f6-42d1-425d-9a33-fef2a9e449e0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:22.440116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f897f2-891b-4a62-aeab-fbbf7812788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:22.440707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f897f2-891b-4a62-aeab-fbbf7812788d None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:22.441008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f897f2-891b-4a62-aeab-fbbf7812788d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:22.441277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f897f2-891b-4a62-aeab-fbbf7812788d None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:22.441940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f897f2-891b-4a62-aeab-fbbf7812788d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:22.443481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f897f2-891b-4a62-aeab-fbbf7812788d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:22.468844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 27/57] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Sep 13 19:36:22 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:24.596852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad78498-4200-4f7a-8da0-5686b6bc80f3 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.597383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad78498-4200-4f7a-8da0-5686b6bc80f3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.597383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad78498-4200-4f7a-8da0-5686b6bc80f3 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.598759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 31/58] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:24.604785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:24.605195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:24.605341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:24.622955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:24.624253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:24.686910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 28/59] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:24.957468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad78498-4200-4f7a-8da0-5686b6bc80f3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:24.979647 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.056473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad78498-4200-4f7a-8da0-5686b6bc80f3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.089374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a760d05-4ec8-45fb-ba61-63f11d09e18c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.090024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760d05-4ec8-45fb-ba61-63f11d09e18c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.090307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760d05-4ec8-45fb-ba61-63f11d09e18c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.090583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a760d05-4ec8-45fb-ba61-63f11d09e18c None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.091694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a760d05-4ec8-45fb-ba61-63f11d09e18c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.093127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a760d05-4ec8-45fb-ba61-63f11d09e18c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.094703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 32/60] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:25.100394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:25.132202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:25.196483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9504779-04e0-4c7c-aff1-61f00ef237d6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:25.229167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657c97aa-c443-4887-a2fa-d936df3bfba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:25.229710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c97aa-c443-4887-a2fa-d936df3bfba2 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:25.230014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c97aa-c443-4887-a2fa-d936df3bfba2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:25.230285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657c97aa-c443-4887-a2fa-d936df3bfba2 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:25.230940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c97aa-c443-4887-a2fa-d936df3bfba2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:25.232375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657c97aa-c443-4887-a2fa-d936df3bfba2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:25.259034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 29/61] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:25 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:28.361892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c56074-dd19-4bca-adf5-78ec40476b00 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.361892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c56074-dd19-4bca-adf5-78ec40476b00 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.361892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c56074-dd19-4bca-adf5-78ec40476b00 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.363703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 33/62] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:28.369904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76819-fe5e-4f6b-8a61-df6c8aee9dc4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.370718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76819-fe5e-4f6b-8a61-df6c8aee9dc4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.371196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c76819-fe5e-4f6b-8a61-df6c8aee9dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.393806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-03c76819-fe5e-4f6b-8a61-df6c8aee9dc4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:28.395326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-03c76819-fe5e-4f6b-8a61-df6c8aee9dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:28.476046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 30/63] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:28.837667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c56074-dd19-4bca-adf5-78ec40476b00 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:28.867442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1d0fa-308e-4b56-8f51-bb2c1c5ff65b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.868084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1d0fa-308e-4b56-8f51-bb2c1c5ff65b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.868584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad1d0fa-308e-4b56-8f51-bb2c1c5ff65b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.870169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 31/64] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:28.875753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341181b-4e96-4c23-a55b-059d1eb4daa1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.876322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341181b-4e96-4c23-a55b-059d1eb4daa1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.876811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a341181b-4e96-4c23-a55b-059d1eb4daa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.897294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a341181b-4e96-4c23-a55b-059d1eb4daa1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:28.898845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a341181b-4e96-4c23-a55b-059d1eb4daa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:28.914408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.948720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:28.964895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c56074-dd19-4bca-adf5-78ec40476b00 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:28.983474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 32/65] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:28.984902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56fa0c-e8d0-4046-a1e3-af0d0f1e1371 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.985388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56fa0c-e8d0-4046-a1e3-af0d0f1e1371 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.985894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a56fa0c-e8d0-4046-a1e3-af0d0f1e1371 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:28.987584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 33/66] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:28.993477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36fe8-b6ac-45ca-b831-6bdf13162ba1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:28.994052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36fe8-b6ac-45ca-b831-6bdf13162ba1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:28.994517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b36fe8-b6ac-45ca-b831-6bdf13162ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.015166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f8b36fe8-b6ac-45ca-b831-6bdf13162ba1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:29.016744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f8b36fe8-b6ac-45ca-b831-6bdf13162ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:29.027284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.028236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.028661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.029074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.045926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:29.057300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:29.058447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:29.059406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:29.061925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 34/67] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:29.067653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.105129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 34/68] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:29.148586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169c59-57f1-4c38-a1e3-17a48fc96e2d None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.149200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169c59-57f1-4c38-a1e3-17a48fc96e2d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.149705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0169c59-57f1-4c38-a1e3-17a48fc96e2d None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.151517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 35/69] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:29.157056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06468f5a-642d-41e9-850a-8d1063e996bc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.157545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06468f5a-642d-41e9-850a-8d1063e996bc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.158110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06468f5a-642d-41e9-850a-8d1063e996bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.192090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-06468f5a-642d-41e9-850a-8d1063e996bc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:29.193285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-06468f5a-642d-41e9-850a-8d1063e996bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:29.208443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55a9b4-6bfe-43d3-8040-9d525a859a4c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.244102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.244930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.245378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.245817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.246613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] RBAC: Adding query filter params (name=member) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:29.247055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.248620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5375980-8ea0-41f7-8411-3fbf0b1a665f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.257321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 35/70] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:29.258702 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357f65e-b348-4d82-9340-06a228abbcee None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.259151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357f65e-b348-4d82-9340-06a228abbcee None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.259548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357f65e-b348-4d82-9340-06a228abbcee None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.260918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 36/71] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:29.265018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357f65e-b348-4d82-9340-06a228abbcee None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.278078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 36/72] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:29.279685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.280044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.280390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.281587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.301453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:29.303008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:29.306069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.375735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 37/73] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:29.381355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357f65e-b348-4d82-9340-06a228abbcee None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.406899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.420973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.421675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.421983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.422257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.436420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.437956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.439983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 37/74] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:29.445863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.515040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.531654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf276b6e-d3d7-4061-ab65-6b79ee1efb26 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.550637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e270c21-2827-4677-bbd1-57e42ceb12f6 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.569516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.570444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.570921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.571409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.581114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.585050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:29.586795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.587414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.587701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.588035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:29.593905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:29.594730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:29.595493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:29.598335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:29.604397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.662037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.663422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.704509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba280d-d154-4de9-adfe-4ca114897020 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.733207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.739265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.739899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.740179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.740454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.751324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:29.756132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:29.757045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:29.757835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 39/76] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:29.758484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.772441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 38/77] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:29.773263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.821960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:29.863895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9c6a14-51b5-4525-8de6-7d86d9a63772 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.875733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37096948-59fc-4b41-aff9-d5c6da57e1f9 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:29.896579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.897206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.897490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.897799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.898930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.900380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.902009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 40/78] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:29.902767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.913170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:29.913713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:29.914019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:29.914291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:29.914954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:29.915253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:29.916598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:29.924527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 39/79] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:29.926601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:29.958858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.001773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595e25b-53ea-41cf-8bac-5402540755ef None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.027442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cdec71-c068-4ebf-862b-28b538511128 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.027571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.035888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.036565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.036844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.037147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.047053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:30.052288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:30.053343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 41/80] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:30.054181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.063545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.064156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.064465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.064741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.065402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.066280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.067601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.075672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 40/81] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.076417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.153463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1351ada7-998d-477a-9228-e1e0edd8e6bf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.166783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.176065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24007d7c-0b3a-4ca2-8e9c-27a9351c176a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.186650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.187277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.187559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.187831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.189069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.191026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.192839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 42/82] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.193408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.214436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.215047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.215325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.215596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.216239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.217956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.244223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 41/83] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:30 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:30.245143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.256146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.309114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5289fa-0cba-4b33-a2dd-01150519cf88 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.350142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.351123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.351604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.352045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.353509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.355155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.356967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 43/84] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.362482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.378616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b1fd18-2009-4303-a5dd-b1345c9e2bf2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.417007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.417611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.417938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.418212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.418889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] RBAC: Adding query filter params (name=member) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.419189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.420574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.431449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 42/85] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.431678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.448619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.486479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093f3424-ba75-402c-b929-545e84fb34c7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.516918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.532061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.533010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.533010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.533222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.535024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.536449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.538891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 44/86] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.539658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.563297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a21baf8-924a-423f-ae21-1839ad262598 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.604853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.605468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.606040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.606040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.606767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=d7a3f9e9eac64a11840e9edae8f10588) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:30.607075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.608517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.616499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 43/87] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/role_assignments?user.id=d7a3f9e9eac64a11840e9edae8f10588&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.616499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.663927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316c457-408a-4156-878c-dfb4a7663914 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.712162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.713159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.713952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.715454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.721858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.790919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:30.804549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.806420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.813571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a853e7-6121-4cdc-a483-a6c524b7f5ef None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:30.894056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:30.894713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:30.894999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:30.895275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:30.903397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:30.905344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:30.907068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 44/88] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:30.907848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:30.949039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 45/89] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:30.950581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.060199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.071882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e036084-1142-4b1f-afb0-ec1a50fb021d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.112309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.112923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.113196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.113465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.145621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bad52-e153-46b3-822a-dfca376e6d9e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.209474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.210169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.210459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.210732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.224567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.226069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.232075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.294708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.296204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.407391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 45/90] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:31.409109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.457548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.467037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.492230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 46/91] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:31.496711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.584623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e052119e-c449-45bc-a232-8f37b6d506c7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.629227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64967d0-36e2-4e8c-9b9b-ce62a7af0de8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.674128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.674677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.675122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.675494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.676653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.678308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.680294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 47/92] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.680875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.686478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.687365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.687824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.688274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.693999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.695722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.697587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 46/93] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.698531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.755327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:31.800291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4e697-b7c6-4647-82d1-6e5a6669ef47 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.840164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:31.840804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:31.841086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:31.841367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] PATH_INFO: `/v3/projects/6c3426d53b7b4211b3da028ef8c97927` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:31.842524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] RBAC: Authorizing `identity:get_project(project_id=6c3426d53b7b4211b3da028ef8c97927)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:31.844242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:31.846182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 48/94] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/projects/6c3426d53b7b4211b3da028ef8c97927 => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:31.847316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:31.964455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da955b-46f4-4f1f-95ad-95d029baa608 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:31.996904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b44dd6-be00-4d07-aeef-df2c9f311483 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.001613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.012809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.020485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.021097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.021374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.021651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.022410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] RBAC: Adding query filter params (role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=d7a3f9e9eac64a11840e9edae8f10588) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.022718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.024150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.034190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 49/95] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/role_assignments?user.id=d7a3f9e9eac64a11840e9edae8f10588&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.036130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:32.073356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.074006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.074291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.074561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.075294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=8e9932faac4043979f160bcfef7916ea) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.075603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.077271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a86aa-9e2c-4c4f-b15f-0775f670633e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.095502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 47/96] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:31 2022] GET /identity/v3/role_assignments?user.id=8e9932faac4043979f160bcfef7916ea&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.208164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ccfa7-e10f-4a18-80b5-6bf948ab69f9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:32.266590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:32.267603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.268082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.268549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.269583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=6c3426d53b7b4211b3da028ef8c97927, user.id=d7a3f9e9eac64a11840e9edae8f10588) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:32.270134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:32.271827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a4d07-c0f5-455d-ab9b-ba08c23f22ef None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:32.278763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 50/97] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:32 2022] GET /identity/v3/role_assignments?user.id=d7a3f9e9eac64a11840e9edae8f10588&scope.project.id=6c3426d53b7b4211b3da028ef8c97927&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:32.869093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6aa55-046c-48e8-b184-8c45aafaf2b7 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.869596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6aa55-046c-48e8-b184-8c45aafaf2b7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.869836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6aa55-046c-48e8-b184-8c45aafaf2b7 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.871196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 48/98] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:32.876900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343058e5-f63f-4975-a4c0-9dcad07046fe None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:32.877368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343058e5-f63f-4975-a4c0-9dcad07046fe None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:32.877465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343058e5-f63f-4975-a4c0-9dcad07046fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:32.912411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-343058e5-f63f-4975-a4c0-9dcad07046fe None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:32.913661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-343058e5-f63f-4975-a4c0-9dcad07046fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:32.979347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:32.996903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 51/99] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:33.044929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42734668-ea6b-415e-9d9d-dee93a7ce6a4 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.045544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42734668-ea6b-415e-9d9d-dee93a7ce6a4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.046059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42734668-ea6b-415e-9d9d-dee93a7ce6a4 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.047650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 49/100] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:33.053048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.053546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.054079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.077188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:33.079329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:33.164770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 52/101] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:33.270192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42734668-ea6b-415e-9d9d-dee93a7ce6a4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.389532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42734668-ea6b-415e-9d9d-dee93a7ce6a4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.430854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.431858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.432331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.432828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.446465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:33.452311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:33.453166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:33.453935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:33.456178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 50/102] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:33.466834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.593952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.617655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d60f20-afa7-468a-b347-f226f933ada3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.637556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.654104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:33.711136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.711682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.711967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.712246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.713187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:33.713496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.714924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0a0ce-5cc5-4d3f-9274-4e089049a28a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.718792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da95852c-4f34-4eb3-8d95-9fb308e82d74 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:33.735956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 53/103] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:33.737239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096928af-b9fe-4e34-9621-3bdff54261a7 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.737529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096928af-b9fe-4e34-9621-3bdff54261a7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.737827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096928af-b9fe-4e34-9621-3bdff54261a7 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.739192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 54/104] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:33.744125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096928af-b9fe-4e34-9621-3bdff54261a7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.765825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabbdba-cb54-43c3-b0b3-f6557d789e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:33.766468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabbdba-cb54-43c3-b0b3-f6557d789e3e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.766748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabbdba-cb54-43c3-b0b3-f6557d789e3e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.767021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabbdba-cb54-43c3-b0b3-f6557d789e3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.768146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabbdba-cb54-43c3-b0b3-f6557d789e3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:33.769800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabbdba-cb54-43c3-b0b3-f6557d789e3e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:33.771386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 51/105] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:33.772559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:33.772843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:33.773115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:33.796901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:33.798133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:33.879873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 52/106] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:33.881326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:33.935574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.006400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096928af-b9fe-4e34-9621-3bdff54261a7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.021187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891e59-4511-4033-8be2-e7264559fe47 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.056331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.057299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.057753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.058163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.058937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.060623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.081308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.082360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.082904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.083315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] PATH_INFO: `/v3/users/7aacc88ac3464eeea5a0d95213e6678d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.097480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 53/107] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:36:33 2022] POST /identity/v3/projects => generated 302 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:34.113611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] RBAC: Authorizing `identity:get_user(user_id=7aacc88ac3464eeea5a0d95213e6678d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.115486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.117618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 55/108] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:33 2022] GET /identity/v3/users/7aacc88ac3464eeea5a0d95213e6678d => generated 289 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.124267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.249929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.270755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacddd2f-7b97-4b63-95cf-604c15fc2779 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.308524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.309148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.309461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.309710 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.399279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.400758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.432007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.468155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.558173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.592934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 54/109] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:34.599953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.612462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de6c9e-a0c8-4b05-bb15-a6efc64be133 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.691958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.692592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.692874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.693150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.713614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:34.719048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:34.719887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:34.720643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 56/110] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:34.731432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.819652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa72ed8-1973-4d1f-8872-5ab96a6d37a6 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.901603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:34.908694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8c6f44-498f-49b3-ab8f-7f71ab343833 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:34.946240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.946436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:34.946907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.946984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:34.947206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.947294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:34.947438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.947715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:34.948539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:34.948724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.949021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:34.950141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.951250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:34.958084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 55/111] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.958959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:34.961101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 57/112] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:34.968458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.099017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.172740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcca602-079e-4a97-a4c3-ec827bcbd964 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.187680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64475dd-5486-4d23-8156-19442aaea090 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.211588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.212207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.212488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.212763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.213485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] RBAC: Adding query filter params (role.id=284c674552164365ab075b703005206e, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=7aacc88ac3464eeea5a0d95213e6678d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:35.213830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.215394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.221579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 56/113] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/role_assignments?user.id=7aacc88ac3464eeea5a0d95213e6678d&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=284c674552164365ab075b703005206e => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:35.275033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.275688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.275964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.276233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.280479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.281999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.284123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 58/114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:34 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:35.290015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.412279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91fda85-9496-4701-b0f3-00e5ca101e44 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.430573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.449331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8731161a-bbe5-42ad-8937-a034737e9efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.449995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731161a-bbe5-42ad-8937-a034737e9efb None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.450888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731161a-bbe5-42ad-8937-a034737e9efb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.451250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8731161a-bbe5-42ad-8937-a034737e9efb None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.452403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731161a-bbe5-42ad-8937-a034737e9efb None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.453939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8731161a-bbe5-42ad-8937-a034737e9efb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.455631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 57/115] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:35 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:35.462075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:35.496352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:35.509306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e609-51cf-46bb-95a5-4070b58fd90c None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.509895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e609-51cf-46bb-95a5-4070b58fd90c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.510344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576e609-51cf-46bb-95a5-4070b58fd90c None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.512290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 58/116] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:35.517697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d401-1939-482b-9efb-5c236fc3da24 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.518068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d401-1939-482b-9efb-5c236fc3da24 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.518821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811d401-1939-482b-9efb-5c236fc3da24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.564236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c811d401-1939-482b-9efb-5c236fc3da24 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:35.567329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c811d401-1939-482b-9efb-5c236fc3da24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:35.672891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a995ad24-6142-4e04-810d-d9214ad9ef79 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:35.736603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 59/117] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:35.743699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639b520-e7ea-4826-aa85-7ccd5d824aff None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.743983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639b520-e7ea-4826-aa85-7ccd5d824aff None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.744256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639b520-e7ea-4826-aa85-7ccd5d824aff None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.745964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 60/118] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:35.751617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb596-18c6-403e-bccf-26b4343c19c4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.752201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb596-18c6-403e-bccf-26b4343c19c4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.752491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb596-18c6-403e-bccf-26b4343c19c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.770483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598abec5-60e2-4d6e-9f92-b3cfbd4f79f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:35.770822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-febdb596-18c6-403e-bccf-26b4343c19c4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:35.771573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598abec5-60e2-4d6e-9f92-b3cfbd4f79f8 None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:35.771995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598abec5-60e2-4d6e-9f92-b3cfbd4f79f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:35.772103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-febdb596-18c6-403e-bccf-26b4343c19c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:35.772497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598abec5-60e2-4d6e-9f92-b3cfbd4f79f8 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d/users/d7a3f9e9eac64a11840e9edae8f10588/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:35.775213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598abec5-60e2-4d6e-9f92-b3cfbd4f79f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=22554735f59a426ea67d528448c2727d, user_id=d7a3f9e9eac64a11840e9edae8f10588, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:35.776675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598abec5-60e2-4d6e-9f92-b3cfbd4f79f8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:35.807632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 59/119] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:35 2022] PUT /identity/v3/projects/22554735f59a426ea67d528448c2727d/users/d7a3f9e9eac64a11840e9edae8f10588/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:35.895434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 61/120] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:36.099423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6668d-e741-4c2f-9d6a-f7966a016475 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.099718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6668d-e741-4c2f-9d6a-f7966a016475 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.099998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6668d-e741-4c2f-9d6a-f7966a016475 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.101384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 60/121] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:36.107753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.108598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.109009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.133695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:36.135089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:36.180867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6668d-e741-4c2f-9d6a-f7966a016475 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.191281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.244149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 62/122] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:36.307292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.328973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6668d-e741-4c2f-9d6a-f7966a016475 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.374315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.374931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.375211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.375482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.390571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:36.395565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.396600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:36.397537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 61/123] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:36.406819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.536450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.550687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2f988d-beb7-4259-9896-d3fb96162796 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.591744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.592627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.593046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.593456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.594281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:36.594719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.596240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.613326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 63/124] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:36.618930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.655779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.722382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a73bd-2dd3-4b53-a2d1-5fa5a9228924 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.723438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc754edb-fe50-4984-9086-123da68c2ebb None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.770246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.771050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.771809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.772192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.783522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:36.788565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:36.789563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:36.790351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:36.790975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 62/125] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:36.792281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.792965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:36.793800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:36.794215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:36.794619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:36.795963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:36.797567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:36.799312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 64/126] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:36.802651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:36.806759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:36.935702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb804494-9f9c-496c-8a3a-dd132fdfa330 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:36.991651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e381ce8-cac3-4e5d-a23c-9ad2f4684ade None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.007072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.007709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.007986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.008257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.010159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.027251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:37.032240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:37.033152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:37.033942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:37.036928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 63/127] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:37.041320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.065674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.066350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.066635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.066912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.067600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.067908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.069328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.091018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 65/128] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.091876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.144445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.181449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de71ac9-dc84-42d1-b223-048005583bd1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.219947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.220845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.221393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.221668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.222896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.224480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.226554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 64/129] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.227200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.265618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e432104c-d4de-450b-866a-c7a7416c54ba None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.303847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.304487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.304766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.305046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.306018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] RBAC: Adding query filter params (name=member) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:37.306326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.308756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe54e7e-de36-4863-a01a-84e64567569b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.320072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 66/130] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.324755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b7a22-5ebf-44b5-a6ac-44c52f43e24f None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.324755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b7a22-5ebf-44b5-a6ac-44c52f43e24f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.324755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b7a22-5ebf-44b5-a6ac-44c52f43e24f None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.324755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 67/131] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:37.324755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b7a22-5ebf-44b5-a6ac-44c52f43e24f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.359953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.383342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1c9f7-7f24-4137-9aef-9090427b5b63 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.434110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.435356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.435751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.436382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.437892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.439815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.441774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 65/132] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.442967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.446615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.459777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b7a22-5ebf-44b5-a6ac-44c52f43e24f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.499812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801f0d-b157-45c5-8f76-ead0b59b669d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.500928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801f0d-b157-45c5-8f76-ead0b59b669d None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.501496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801f0d-b157-45c5-8f76-ead0b59b669d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.502086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801f0d-b157-45c5-8f76-ead0b59b669d None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d/users/d7a3f9e9eac64a11840e9edae8f10588/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.505070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3801f0d-b157-45c5-8f76-ead0b59b669d None admin] RBAC: Authorizing `identity:create_grant(project_id=22554735f59a426ea67d528448c2727d, user_id=d7a3f9e9eac64a11840e9edae8f10588, role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.506766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3801f0d-b157-45c5-8f76-ead0b59b669d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.526780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 68/133] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:37 2022] PUT /identity/v3/projects/22554735f59a426ea67d528448c2727d/users/d7a3f9e9eac64a11840e9edae8f10588/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:37.528788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.529308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.529871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.552208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:37.553907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:37.574187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9554-c413-4969-8a3e-33ee473d5a4e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.637286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.639422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.643183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.644737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.646359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.648302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.654534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 66/134] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.657622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.679779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 69/135] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:37.681235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.737786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.799539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:37.847313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ab0cb8-32c8-4f14-83d6-b8240978a7bc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:37.898975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:37.899640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:37.899925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:37.900203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:37.901994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:37.903588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:37.905422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 67/136] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:37.912153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:37.927981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c570e-202e-4f22-8ea4-0d8403fc096d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.010973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.011686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.011964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.012287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] PATH_INFO: `/v3/projects/6c3426d53b7b4211b3da028ef8c97927` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.018456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] RBAC: Authorizing `identity:get_project(project_id=6c3426d53b7b4211b3da028ef8c97927)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.019918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.021602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 70/137] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:37 2022] GET /identity/v3/projects/6c3426d53b7b4211b3da028ef8c97927 => generated 312 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.028518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.091817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b5892-d8a6-4fad-a583-3c02c0c48fe6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.140114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f302db-cb00-40d4-9e56-55ca4556c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.140896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f302db-cb00-40d4-9e56-55ca4556c87c None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f302db-cb00-40d4-9e56-55ca4556c87c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.141571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f302db-cb00-40d4-9e56-55ca4556c87c None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d/users/8e9932faac4043979f160bcfef7916ea/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.143537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f302db-cb00-40d4-9e56-55ca4556c87c None admin] RBAC: Authorizing `identity:create_grant(project_id=22554735f59a426ea67d528448c2727d, user_id=8e9932faac4043979f160bcfef7916ea, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.144992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f302db-cb00-40d4-9e56-55ca4556c87c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.156812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.161031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 68/138] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:37 2022] PUT /identity/v3/projects/22554735f59a426ea67d528448c2727d/users/8e9932faac4043979f160bcfef7916ea/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:38.167013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.170704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54167fc-cad3-4bb1-8196-731534dc38cb None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.172141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54167fc-cad3-4bb1-8196-731534dc38cb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.172141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54167fc-cad3-4bb1-8196-731534dc38cb None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.172839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 69/139] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:38.174533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54167fc-cad3-4bb1-8196-731534dc38cb None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.202124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926d06f-e7c2-4cae-b7b5-b8caeb3cf3cc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.296669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da071a67-4c87-4022-ac43-1695e9df4f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.297448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da071a67-4c87-4022-ac43-1695e9df4f73 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.297691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da071a67-4c87-4022-ac43-1695e9df4f73 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.298010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da071a67-4c87-4022-ac43-1695e9df4f73 None admin] PATH_INFO: `/v3/projects/6c3426d53b7b4211b3da028ef8c97927/users/d7a3f9e9eac64a11840e9edae8f10588/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.300041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da071a67-4c87-4022-ac43-1695e9df4f73 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c3426d53b7b4211b3da028ef8c97927, user_id=d7a3f9e9eac64a11840e9edae8f10588, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.301542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da071a67-4c87-4022-ac43-1695e9df4f73 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.328373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54167fc-cad3-4bb1-8196-731534dc38cb None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.342275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 71/140] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:38 2022] PUT /identity/v3/projects/6c3426d53b7b4211b3da028ef8c97927/users/d7a3f9e9eac64a11840e9edae8f10588/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:38.343456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f75af-2899-46e7-acef-dd0d2b02fddf None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.343761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f75af-2899-46e7-acef-dd0d2b02fddf None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.344041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f75af-2899-46e7-acef-dd0d2b02fddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.406105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1e7f75af-2899-46e7-acef-dd0d2b02fddf None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:38.407699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1e7f75af-2899-46e7-acef-dd0d2b02fddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:38.414501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.415211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.415499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.415776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.417905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.419555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.421628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 70/141] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:38.427685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:38.559806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:38.573785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 72/142] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:38.601629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce3d013-b105-4b12-b9b6-446023f86e67 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:38.640850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bcd773-fd16-4d61-914b-878bc553a474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:38.641755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd773-fd16-4d61-914b-878bc553a474 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.642187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd773-fd16-4d61-914b-878bc553a474 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.642603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bcd773-fd16-4d61-914b-878bc553a474 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.643404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcd773-fd16-4d61-914b-878bc553a474 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:38.645100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bcd773-fd16-4d61-914b-878bc553a474 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:38.671924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 71/143] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:38 2022] POST /identity/v3/users => generated 283 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:36:38.847637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eda6e-2439-42f4-9c7f-6e07eb2dfae3 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.847967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eda6e-2439-42f4-9c7f-6e07eb2dfae3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.848215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eda6e-2439-42f4-9c7f-6e07eb2dfae3 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.849571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 73/144] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:38.855797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:38.856342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:38.856934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:38.883210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:38.884806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:38.956251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 72/145] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:38.959628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eda6e-2439-42f4-9c7f-6e07eb2dfae3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.079109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.106532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eda6e-2439-42f4-9c7f-6e07eb2dfae3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.156517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.157160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.157438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.157707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.171308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:39.176153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:39.177039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:39.177942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 74/146] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:39.183116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.255564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.310117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2ee3e-5b20-48ef-bd66-89816f2877a0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.349484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.350511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.350991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.351450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.352331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:39.353458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.355085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.364152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 73/147] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.369879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.376229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.555031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721a6cb3-66ae-49ad-9064-c66420f7e0b0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.584367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.594838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.595748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.596237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.596695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.597606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f122bb-299a-466d-993f-f7376965214f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.609589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:39.615371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:39.616198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:39.616968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:39.619653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 74/148] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:39.625925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.691001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.691643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.691925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.692202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] PATH_INFO: `/v3/users/7aacc88ac3464eeea5a0d95213e6678d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.693469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] RBAC: Authorizing `identity:get_user(user_id=7aacc88ac3464eeea5a0d95213e6678d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.694848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.696982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 75/149] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/users/7aacc88ac3464eeea5a0d95213e6678d => generated 289 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:39.703018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.711452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.741761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1aafa-8af1-4602-8cee-0d8173a5691b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.780975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.782084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.782571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.783036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.783907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:39.784453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.786412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:39.798122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 75/150] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:39.804641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:39.898204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:39.924464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad785a4-1644-4760-aa7d-c0fb39210298 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.987784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e7f4e-501c-4d59-b79d-a7561c83b6fd None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:39.991306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:39.991900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:39.997781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:39.997781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:39.997781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:39.997781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.005989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 76/151] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.006946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.116068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.116985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.117952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.118594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.120819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.128070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.130148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 76/152] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:39 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.144647 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.164955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64651f60-9c22-4923-b5a9-d1dee80ee5a9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.167679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.222174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.222897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.223178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.223440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.326120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.384014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.385512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.412603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993b27e-9c82-4962-90f7-9cb4249b5a4a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.513019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.515453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.515958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.516438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d/users/7aacc88ac3464eeea5a0d95213e6678d/roles/284c674552164365ab075b703005206e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.519280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] RBAC: Authorizing `identity:create_grant(project_id=22554735f59a426ea67d528448c2727d, user_id=7aacc88ac3464eeea5a0d95213e6678d, role_id=284c674552164365ab075b703005206e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.520991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.556486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 77/153] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:40 2022] PUT /identity/v3/projects/22554735f59a426ea67d528448c2727d/users/7aacc88ac3464eeea5a0d95213e6678d/roles/284c674552164365ab075b703005206e => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:36:40.640393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 77/154] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:40.651089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.824774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee2b6c3-7591-46a0-b95e-8bcb428cb492 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:40.877761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:40.883592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52986b9-5524-4242-b01e-5110aa1e3b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:40.884534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52986b9-5524-4242-b01e-5110aa1e3b07 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:40.885189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52986b9-5524-4242-b01e-5110aa1e3b07 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:40.885698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52986b9-5524-4242-b01e-5110aa1e3b07 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:40.887095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52986b9-5524-4242-b01e-5110aa1e3b07 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:40.888767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52986b9-5524-4242-b01e-5110aa1e3b07 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:40.891053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 78/155] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:40.896816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:40.908024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.082347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9006006-e271-43c2-99f3-98b8ca7d9344 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:41.125076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:41.125684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.125988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.126260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.126993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=d7a3f9e9eac64a11840e9edae8f10588) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:41.127304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:41.128728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee23131-5635-47f5-ab49-1f7889e800f8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:41.135612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 78/156] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:40 2022] GET /identity/v3/role_assignments?user.id=d7a3f9e9eac64a11840e9edae8f10588&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:41.149579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bace2-b889-4517-bb16-9bfd0f40d9cc None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.150188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bace2-b889-4517-bb16-9bfd0f40d9cc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.150653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794bace2-b889-4517-bb16-9bfd0f40d9cc None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.152216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 79/157] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:41.158090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:41.158412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:41.158641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:41.186790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:41.188022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:41.265626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:41.271228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 79/158] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:41.822701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bace2-b889-4517-bb16-9bfd0f40d9cc None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:41.983752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794bace2-b889-4517-bb16-9bfd0f40d9cc None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.025370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.026044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.026321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.026590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.040936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:42.045814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:42.046686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:42.047496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 80/159] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:42.052825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.231314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109c4ed5-b9c7-4edb-9d28-5bf24c23e664 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.284478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.285098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.285372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.285636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.286358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:42.286650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.288596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da229994-77e3-4be0-b4ab-2878e6d83f4b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.299091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 80/160] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:42.305606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.360929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83576e16-7400-40ef-8a7b-6daaf293c911 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.361243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83576e16-7400-40ef-8a7b-6daaf293c911 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.361473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83576e16-7400-40ef-8a7b-6daaf293c911 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.363064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 81/161] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:42.368332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086fa87-11c0-4d5f-829c-d6cb0ae8d20f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.368606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086fa87-11c0-4d5f-829c-d6cb0ae8d20f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.368880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6086fa87-11c0-4d5f-829c-d6cb0ae8d20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.396595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.409569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6086fa87-11c0-4d5f-829c-d6cb0ae8d20f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:42.411534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6086fa87-11c0-4d5f-829c-d6cb0ae8d20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:42.477505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce589b47-2083-4de9-8985-276cdb490a34 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.514971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 82/162] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 19:36:42.516323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d289e-6422-45e9-857c-426db2022b06 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.516607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d289e-6422-45e9-857c-426db2022b06 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.516880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84d289e-6422-45e9-857c-426db2022b06 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.518302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 83/163] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:42.520602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.525497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.525815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.526139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.548442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:42.550026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:42.566449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297505-207c-419f-9490-918aac4519e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.567102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297505-207c-419f-9490-918aac4519e3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.567380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297505-207c-419f-9490-918aac4519e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.567658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72297505-207c-419f-9490-918aac4519e3 None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.569384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72297505-207c-419f-9490-918aac4519e3 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.570889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72297505-207c-419f-9490-918aac4519e3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:42.572489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 81/164] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:42.579464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297505-207c-419f-9490-918aac4519e3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:42.682367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 84/165] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:42.764594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72297505-207c-419f-9490-918aac4519e3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:42.799919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:42.810366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:42.811248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:42.811738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:42.812206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:42.906859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:42.909121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.034381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 82/166] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:43.040586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.103227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.159650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.191935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ca8b4-151e-48c0-9731-0a477567d62e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.223125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.255683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.256332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.256742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.256935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.258397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.259900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.261905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 85/167] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.262868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.323559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688a9d90-9afb-48a0-bfe0-0951ec90279d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.388967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.389527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.389833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.390103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.398225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed9861-7101-49ce-b6fd-4db6ce04d220 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.413308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:43.421669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:43.422732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:43.423503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:43.428735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 83/168] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:43.430621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.469166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e01d89-adf9-481d-a695-7b5aaffbefd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.469849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e01d89-adf9-481d-a695-7b5aaffbefd6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.470124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e01d89-adf9-481d-a695-7b5aaffbefd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.470445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e01d89-adf9-481d-a695-7b5aaffbefd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.487976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e01d89-adf9-481d-a695-7b5aaffbefd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-e5e01d89-adf9-481d-a695-7b5aaffbefd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:43.493427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:43.494343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:43.495133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:43.501435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 86/169] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:43.502747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da3aee-f14c-4f21-b33c-4b9612679503 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.503024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da3aee-f14c-4f21-b33c-4b9612679503 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.503302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da3aee-f14c-4f21-b33c-4b9612679503 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.504563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 87/170] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:43.509643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da3aee-f14c-4f21-b33c-4b9612679503 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.541305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.555582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6633b65-7bb9-47f1-a3f0-5075658fc962 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.598426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.600082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.600563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.601035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.602682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] RBAC: Adding query filter params (role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=d7a3f9e9eac64a11840e9edae8f10588) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.603900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.606359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10e399-3609-4232-ac4a-5a1cd4ee0dc9 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.614095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.623474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 84/171] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/role_assignments?user.id=d7a3f9e9eac64a11840e9edae8f10588&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.624768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.625057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.625332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.667160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:43.668319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:43.719910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da3aee-f14c-4f21-b33c-4b9612679503 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.756688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 85/172] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:43.757571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.810543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.811516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.812007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.812483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.813366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.813922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.815511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c6723-8626-4b12-b5c4-12e98bdc4471 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.833575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 88/173] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.835225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be97ec1-be90-4d4a-bbe5-d0fa36e36af6 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.835737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be97ec1-be90-4d4a-bbe5-d0fa36e36af6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.836276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be97ec1-be90-4d4a-bbe5-d0fa36e36af6 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.837900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 89/174] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:43.847292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be97ec1-be90-4d4a-bbe5-d0fa36e36af6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:43.851451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:43.873056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689b0c6c-8d84-49da-b8be-98a22fb4d9ee None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:43.910623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:43.911812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.912079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.912350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.913035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:43.913336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:43.914754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d3b9c-679f-48db-9ec1-85c196e9a92c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:43.924524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 86/175] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:43.925677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:43.925991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:43.926265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:43.947007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:43.948307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:44.023111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 87/176] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:44.023965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.107069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.131727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be97ec1-be90-4d4a-bbe5-d0fa36e36af6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.137448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf26e6-ea5a-4ad2-9617-ddab592100e5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.155232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.174985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.175558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.175835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.176154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] PATH_INFO: `/v3/users/8e9932faac4043979f160bcfef7916ea` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.177397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] RBAC: Authorizing `identity:get_user(user_id=8e9932faac4043979f160bcfef7916ea)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.181751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.181751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 88/177] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/users/8e9932faac4043979f160bcfef7916ea => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.184797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.222450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.223702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.224147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.224556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] PATH_INFO: `/v3/users/d7a3f9e9eac64a11840e9edae8f10588` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.225908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] RBAC: Authorizing `identity:get_user(user_id=d7a3f9e9eac64a11840e9edae8f10588)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.227471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.229483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 90/178] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:43 2022] GET /identity/v3/users/d7a3f9e9eac64a11840e9edae8f10588 => generated 275 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.230422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.294900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5c2903-ba14-40b2-a13a-253ec8168847 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.338208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40492a42-cbb5-415e-bb63-ff1339dfe55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.338856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492a42-cbb5-415e-bb63-ff1339dfe55b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.339391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492a42-cbb5-415e-bb63-ff1339dfe55b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.339820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40492a42-cbb5-415e-bb63-ff1339dfe55b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.351762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40492a42-cbb5-415e-bb63-ff1339dfe55b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-40492a42-cbb5-415e-bb63-ff1339dfe55b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:44.357091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:44.358187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:44.359179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:44.361147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 89/179] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:44.362588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3af473-584f-4253-bf3d-9400b869df3b None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.363513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3af473-584f-4253-bf3d-9400b869df3b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.364488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3af473-584f-4253-bf3d-9400b869df3b None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.365986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 90/180] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:44.366832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3af473-584f-4253-bf3d-9400b869df3b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.370100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.396054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edaab4b-62d9-4331-9b11-fa63049f23b8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.433197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.433716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.434018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.434289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.482743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.489366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3af473-584f-4253-bf3d-9400b869df3b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.506884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.508483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.527444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d3f51-794b-4849-8905-a96db095a24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.528055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3f51-794b-4849-8905-a96db095a24d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.528333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3f51-794b-4849-8905-a96db095a24d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.528605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d3f51-794b-4849-8905-a96db095a24d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.597261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d3f51-794b-4849-8905-a96db095a24d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.598797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d3f51-794b-4849-8905-a96db095a24d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.600669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.611429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 91/181] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:44.613191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.700912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.709170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 91/182] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:44.710719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.711004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.711275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.729391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:44.729598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6b91a-772b-45b5-bac4-b896f521d138 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.730686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:44.763205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.763800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.764076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.764350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.765010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:44.765310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.766712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.774689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 92/183] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.775453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.798611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 92/184] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:44.802728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.841157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.883478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c698b3-8b5d-4349-8606-e4b51e0853e1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.911218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61404000-0acf-4b29-957b-389016c6aad2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:44.918534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.919205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.919489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.919765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.920868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.922340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.924067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 93/185] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.927364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:44.940374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:44.950639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf413a-02c9-4dd5-8a92-95d90da156fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:44.951381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf413a-02c9-4dd5-8a92-95d90da156fe None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.951689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf413a-02c9-4dd5-8a92-95d90da156fe None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.952502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf413a-02c9-4dd5-8a92-95d90da156fe None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.954179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf413a-02c9-4dd5-8a92-95d90da156fe None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:44.956366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf413a-02c9-4dd5-8a92-95d90da156fe None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:44.958474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 93/186] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:44.963024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68737-2619-4dd1-ae34-9db7c2a83807 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:44.963312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68737-2619-4dd1-ae34-9db7c2a83807 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:44.963639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f68737-2619-4dd1-ae34-9db7c2a83807 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:44.964906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 94/187] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:44.969512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68737-2619-4dd1-ae34-9db7c2a83807 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.049103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a493dbe0-c5e1-4eb6-9a30-9ee062025d37 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.072367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.076671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f68737-2619-4dd1-ae34-9db7c2a83807 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.085589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.086244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.086520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.086815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] PATH_INFO: `/v3/projects/6c3426d53b7b4211b3da028ef8c97927` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.087923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] RBAC: Authorizing `identity:get_project(project_id=6c3426d53b7b4211b3da028ef8c97927)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.089348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.095638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 94/188] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/projects/6c3426d53b7b4211b3da028ef8c97927 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.095638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.113418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.114081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.114358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.114630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] PATH_INFO: `/v3/users/3d53df0fb9344a9ba064e57d504b6b44` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.127265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] RBAC: Authorizing `identity:get_user(user_id=3d53df0fb9344a9ba064e57d504b6b44)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.128713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.130378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 95/189] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:44 2022] GET /identity/v3/users/3d53df0fb9344a9ba064e57d504b6b44 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.134807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.169176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.191428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a793c3b0-f0fd-4cf0-ac49-f703313fd8bc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.224922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba15069-426f-4b85-8e38-25af9996b771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.225495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba15069-426f-4b85-8e38-25af9996b771 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.225807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba15069-426f-4b85-8e38-25af9996b771 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.226011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba15069-426f-4b85-8e38-25af9996b771 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.226653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba15069-426f-4b85-8e38-25af9996b771 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.228057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba15069-426f-4b85-8e38-25af9996b771 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.232098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdf79a8-ba99-4790-b053-1e31cecee336 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.247754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 95/190] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:45 2022] POST /identity/v3/users => generated 297 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:45.249330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.249620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.249912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.268353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.269234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.269901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:45.270318 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.270721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.271031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:45.271763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=8e9932faac4043979f160bcfef7916ea) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:45.272305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.273962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.280937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 96/191] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/role_assignments?user.id=8e9932faac4043979f160bcfef7916ea&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.282056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.362266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 96/192] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:45.363541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.379766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.477991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.534748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b06def-56c7-476a-938d-beef764c6c0e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.536830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cca59e-b354-4997-9e93-99e65fe8c9a7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.575195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.576115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.576374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.576644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.577628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=6c3426d53b7b4211b3da028ef8c97927, user.id=d7a3f9e9eac64a11840e9edae8f10588) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:45.578059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.579479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.580342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.580980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.581252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.581521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.586898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 97/193] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/role_assignments?user.id=d7a3f9e9eac64a11840e9edae8f10588&scope.project.id=6c3426d53b7b4211b3da028ef8c97927&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:45.589902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.605072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:45.610125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:45.611094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:45.611925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 97/194] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:45.619865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.749789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.750183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb9ca0-2943-47af-bf21-3bd5925d80e8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.779772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca38d2a4-65b1-4f7a-b3d5-01b3c1bb8ace None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:45.793226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:45.806009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.807112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.807454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.807796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.826605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:45.827777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:45.828224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:45.830880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:45.831806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:45.832112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.833528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.849339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 98/195] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:45.850087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:45.894854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:45.896630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:45.976902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e22a47-c9dd-4a68-bdfa-6f8682df8099 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.021809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.023474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.024047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.024487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.026410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 98/196] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:46.027728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.046212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:46.057844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:46.060729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:46.061509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:46.063227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 99/197] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:46.064810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.068184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.075065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.167969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e479438-2846-453e-98ee-8f81c03f88a3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.192120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91373cd0-4954-41a7-963e-3ca591215649 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.204402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.205301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.205866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.206398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] PATH_INFO: `/v3/users/3d53df0fb9344a9ba064e57d504b6b44` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.207777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] RBAC: Authorizing `identity:get_user(user_id=3d53df0fb9344a9ba064e57d504b6b44)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.209496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.211728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 99/198] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/users/3d53df0fb9344a9ba064e57d504b6b44 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.212674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.232735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.233330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.233608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.233913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.235088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.236534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.239236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 100/199] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.240150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.311464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.327602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.336991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc00237-0ad0-4060-a64f-9e914cbc4c65 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.345349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c779ef88-2945-40b4-8d19-c560a40f1edf None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.377094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.378145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.378635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.379069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.380160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] RBAC: Adding query filter params (name=reader) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:46.380627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.381037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.381754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.382112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.382453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.382939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.392338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 100/200] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.393556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.453382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.455033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.506560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551110dd-637c-4e22-8c93-a5af9893b31b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.544395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.545389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.546012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.546506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.546950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.548515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=3d53df0fb9344a9ba064e57d504b6b44) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:46.549020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.551074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.558668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 101/201] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/role_assignments?user.id=3d53df0fb9344a9ba064e57d504b6b44&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.559743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.587826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.590291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 101/202] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:46.601520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.708899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a82ca-15b0-49cc-bf5a-eaa88ae45f5c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.756377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.762371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.762658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.762937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.764258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.765642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a5e0f2-37f8-4a90-99f4-954be373bb0d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:46.765797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.767484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 102/203] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:46.777769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.844282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:46.845347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:46.847162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:46.847660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] PATH_INFO: `/v3/users/7aacc88ac3464eeea5a0d95213e6678d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:46.853425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] RBAC: Authorizing `identity:get_user(user_id=7aacc88ac3464eeea5a0d95213e6678d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:46.855082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:46.856858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 102/204] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/users/7aacc88ac3464eeea5a0d95213e6678d => generated 289 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:46.859721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:46.863263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:46.984513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23071ed2-2e61-4c37-8432-d1ccbe778d0f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.014124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.030394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c356d9aa-cd77-4dc2-b8aa-874e4bba3f17 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.051074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.051686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.051959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.052228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.052944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] RBAC: Adding query filter params (role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=3d53df0fb9344a9ba064e57d504b6b44) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.053247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.054688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.068833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 103/205] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/role_assignments?user.id=3d53df0fb9344a9ba064e57d504b6b44&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 259 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.083431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.084062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.084340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.084619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.183800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.185629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.297379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.318773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 103/206] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:47.324007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.471455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633231dd-2554-4a44-96c6-859e7f1f9f8d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.475529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:47.517639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11c651-cab7-4245-9b7c-e2f3f44a69f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.518308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c651-cab7-4245-9b7c-e2f3f44a69f3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.518585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c651-cab7-4245-9b7c-e2f3f44a69f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.518853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11c651-cab7-4245-9b7c-e2f3f44a69f3 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.520029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c651-cab7-4245-9b7c-e2f3f44a69f3 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.521527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11c651-cab7-4245-9b7c-e2f3f44a69f3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.523483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 104/207] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:47.529141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:47.701639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b2ceae-e48c-4155-86bd-74dc024516c2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:47.745181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:47.745815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:47.746095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:47.746365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:47.747096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] RBAC: Adding query filter params (role.id=284c674552164365ab075b703005206e, scope.project.id=22554735f59a426ea67d528448c2727d, user.id=7aacc88ac3464eeea5a0d95213e6678d) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:47.747400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:47.748830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996efcc3-faf4-48b5-a019-d1c4ab536a89 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:47.758069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 104/208] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:47 2022] GET /identity/v3/role_assignments?user.id=7aacc88ac3464eeea5a0d95213e6678d&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=284c674552164365ab075b703005206e => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.065611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce643df-8f5e-448e-8fc3-9db9f639326d None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.066150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce643df-8f5e-448e-8fc3-9db9f639326d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.066150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce643df-8f5e-448e-8fc3-9db9f639326d None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.067695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 105/209] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:48.073281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9be6-73e5-4afa-aa70-cafefb8a8bb2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.073924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9be6-73e5-4afa-aa70-cafefb8a8bb2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.074505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c9be6-73e5-4afa-aa70-cafefb8a8bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.096773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a36c9be6-73e5-4afa-aa70-cafefb8a8bb2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:48.098355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a36c9be6-73e5-4afa-aa70-cafefb8a8bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:48.162125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.178530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 105/210] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:48.389570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727a861-b9fb-47ce-8006-306fd667dbc0 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.389948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727a861-b9fb-47ce-8006-306fd667dbc0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.390124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c727a861-b9fb-47ce-8006-306fd667dbc0 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.391692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 106/211] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:48.397582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.397927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.398199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.421845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:48.423031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:48.483903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727a861-b9fb-47ce-8006-306fd667dbc0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.502615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 106/212] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:48.595603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c727a861-b9fb-47ce-8006-306fd667dbc0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.630894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.631536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.631813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.632087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.644606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:48.649381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:48.650287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:48.651087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 107/213] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:48.658035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.759565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca1bc90-33a2-4974-984a-f282dea2a1d1 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.775326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.776193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.795848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.796762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.797230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.797712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.799120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:48.799611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.801145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.809838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 107/214] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:48.816235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.818558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:48.903366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d599b7ab-e655-4e62-aa0b-09d1b85653be None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.930228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49095a-2a7b-4a5a-8836-8c857145b54c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:48.940957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.941976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.942485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.942994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.944352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.946219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.952934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 108/215] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:48.954172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:48.969229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:48.971296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:48.971296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:48.971296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] PATH_INFO: `/v3/users/82e34f5c7ffc437fa88c6610b93cb150` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:48.990183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] RBAC: Authorizing `identity:get_user(user_id=82e34f5c7ffc437fa88c6610b93cb150)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:48.991676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:48.997841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 108/216] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/users/82e34f5c7ffc437fa88c6610b93cb150 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:49.000646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.038065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.075779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.080501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e5daaa-4158-44a1-a87f-d551aebf0467 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.113665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd376f5-0f3e-4b37-9fe1-f284bd167957 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.119249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5672e2da-f629-470a-b4c0-074932070ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.120111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e2da-f629-470a-b4c0-074932070ed4 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.120594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e2da-f629-470a-b4c0-074932070ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.121118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5672e2da-f629-470a-b4c0-074932070ed4 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.121986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e2da-f629-470a-b4c0-074932070ed4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.123724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5672e2da-f629-470a-b4c0-074932070ed4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.149106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a603503-49c6-4e75-8ae7-66d02d323f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.149710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a603503-49c6-4e75-8ae7-66d02d323f0f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.150034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a603503-49c6-4e75-8ae7-66d02d323f0f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.156784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a603503-49c6-4e75-8ae7-66d02d323f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.156948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 109/217] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:48 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:49.156948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e1cc2-d082-4529-a426-7495ef8daddd None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.156948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e1cc2-d082-4529-a426-7495ef8daddd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.156948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641e1cc2-d082-4529-a426-7495ef8daddd None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.157896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 110/218] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:49.163737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.164039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.165249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.186135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:49.187354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:49.256590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a603503-49c6-4e75-8ae7-66d02d323f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.258453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a603503-49c6-4e75-8ae7-66d02d323f0f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.271296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 111/219] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:49.294822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.370302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 109/220] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:49.376019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.472686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.491979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df8753-21fa-4196-87ea-878c337ded85 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.506765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2ec3-217a-416e-a2f7-57d7cab4ad45 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.507058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2ec3-217a-416e-a2f7-57d7cab4ad45 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.507363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af2ec3-217a-416e-a2f7-57d7cab4ad45 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.508847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 110/221] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:49.514049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.514330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.514601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.536279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:49.537458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:49.538825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.539470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.539746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.540015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.541785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.542625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.544570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 112/222] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:49.550834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.629567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 111/223] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:49.630435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.670841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faae15ff-bf81-4e14-be8e-170a00a7c72f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.705817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:49.736400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.736931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.737210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.737483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.738239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=82e34f5c7ffc437fa88c6610b93cb150) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:49.738546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:49.740227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:49.750951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 113/224] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/role_assignments?user.id=82e34f5c7ffc437fa88c6610b93cb150&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:49.777099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea64c1c-1988-4ace-8669-08a6bcae6b36 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:49.835730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:49.836424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:49.836646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:49.836914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:49.858743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:49.864027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:49.865166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:49.866171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 112/225] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:49.873223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:49.905474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.004034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b4068-514c-4f5a-aace-8fd56f593f7e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.012149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.038907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.039905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.040194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.040471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.041146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:50.041455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.042881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.051243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 114/226] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.056723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.152232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.186588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57a3338-a8c2-4ad5-bb95-8b53e44fc488 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.223877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a5be8e-1eb2-459e-8ad4-b8cae4be97df None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.230676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.231329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.231609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.231938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] PATH_INFO: `/v3/users/3d53df0fb9344a9ba064e57d504b6b44` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.233175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] RBAC: Authorizing `identity:get_user(user_id=3d53df0fb9344a9ba064e57d504b6b44)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.234804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.236411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 115/227] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/users/3d53df0fb9344a9ba064e57d504b6b44 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:50.242140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.277164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.290910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.291541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.291822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.292249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.317677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:50.326859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:50.327728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:50.328552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 113/228] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:36:50.335516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.428455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c1749-a93c-454a-a7d9-2403fbc79b48 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.500814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.501916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.502411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.502918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.504935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.506628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.509079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 116/229] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.514826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.519956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9c69f9-4e5f-4260-9eed-aafbcc97db0d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.545931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.585635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.586649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.587186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.587646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.588497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:50.589038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.590684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.605239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 114/230] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.612103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.706112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.715475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a76bd-17aa-44a1-b78c-40fab453eb33 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.771973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa2955c-bfb5-4857-8ae6-19db1c77d241 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:50.790117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.790784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.791058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.791331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/3d53df0fb9344a9ba064e57d504b6b44/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.795215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=69a6097d627b4c9da43f6fb3ca86f1ca, user_id=3d53df0fb9344a9ba064e57d504b6b44, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.797382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.820908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 117/231] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:50 2022] PUT /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/3d53df0fb9344a9ba064e57d504b6b44/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:50.848190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:50.848902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:50.849181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:50.849500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] PATH_INFO: `/v3/users/3d53df0fb9344a9ba064e57d504b6b44` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:50.850800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] RBAC: Authorizing `identity:get_user(user_id=3d53df0fb9344a9ba064e57d504b6b44)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:50.852312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:50.854382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 115/232] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/users/3d53df0fb9344a9ba064e57d504b6b44 => generated 283 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:50.863024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:50.961808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:50.994408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb8e395-95c0-47db-ac38-1ef71c0f93c4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.033656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d40a511-32f3-4d74-ab0b-f7f9511a73f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.034332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40a511-32f3-4d74-ab0b-f7f9511a73f6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.034605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40a511-32f3-4d74-ab0b-f7f9511a73f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.034875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d40a511-32f3-4d74-ab0b-f7f9511a73f6 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.036074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40a511-32f3-4d74-ab0b-f7f9511a73f6 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.037602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d40a511-32f3-4d74-ab0b-f7f9511a73f6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.039581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 118/233] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:50 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:51.045255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:51.150893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56dbe55-2931-4c3d-94d1-1e48b59a31cb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:51.165438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:51.194173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fafcee-f1dc-4fc4-9ad1-0ab491da8bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:51.194826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fafcee-f1dc-4fc4-9ad1-0ab491da8bac None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.195100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fafcee-f1dc-4fc4-9ad1-0ab491da8bac None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.195400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fafcee-f1dc-4fc4-9ad1-0ab491da8bac None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/3d53df0fb9344a9ba064e57d504b6b44/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.201332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fafcee-f1dc-4fc4-9ad1-0ab491da8bac None admin] RBAC: Authorizing `identity:create_grant(project_id=69a6097d627b4c9da43f6fb3ca86f1ca, user_id=3d53df0fb9344a9ba064e57d504b6b44, role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:51.203307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fafcee-f1dc-4fc4-9ad1-0ab491da8bac None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:51.232691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 116/234] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:51 2022] PUT /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/3d53df0fb9344a9ba064e57d504b6b44/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:51.878775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea457-73d7-468e-801e-2058c5207cdc None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.879286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea457-73d7-468e-801e-2058c5207cdc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.879360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ea457-73d7-468e-801e-2058c5207cdc None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.880860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 119/235] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:51.886426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495819c-bdb4-4fc3-8b29-a5446542d6ab None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:51.886870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495819c-bdb4-4fc3-8b29-a5446542d6ab None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:51.887064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6495819c-bdb4-4fc3-8b29-a5446542d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:51.910319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6495819c-bdb4-4fc3-8b29-a5446542d6ab None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:51.911545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6495819c-bdb4-4fc3-8b29-a5446542d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:52.006833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 117/236] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:52.120308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71060a48-333f-4ab5-ac9a-dfbc486a4a5b None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.120489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71060a48-333f-4ab5-ac9a-dfbc486a4a5b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.120770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71060a48-333f-4ab5-ac9a-dfbc486a4a5b None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.122144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 120/237] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:52.128095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.128617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.129133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.158428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:52.159950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:52.257996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 118/238] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:52.479510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71060a48-333f-4ab5-ac9a-dfbc486a4a5b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.572890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.609685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71060a48-333f-4ab5-ac9a-dfbc486a4a5b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.622625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.659250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b050c37-83cf-4ff7-ae11-8afbbf8ea723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.660407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b050c37-83cf-4ff7-ae11-8afbbf8ea723 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.660882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b050c37-83cf-4ff7-ae11-8afbbf8ea723 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.661322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b050c37-83cf-4ff7-ae11-8afbbf8ea723 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.675142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b050c37-83cf-4ff7-ae11-8afbbf8ea723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1b050c37-83cf-4ff7-ae11-8afbbf8ea723 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:52.691299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:52.692213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:52.692995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:52.695325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 121/239] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:52.696930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286015c-ee7d-43c5-b831-aefb2eb9983a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.697416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286015c-ee7d-43c5-b831-aefb2eb9983a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.698159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7286015c-ee7d-43c5-b831-aefb2eb9983a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.699596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 122/240] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:52.704952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286015c-ee7d-43c5-b831-aefb2eb9983a None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.707298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd781be-6c47-4e5d-84c2-b99be3458886 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.709286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.745177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efa7019-3140-45cd-b4bf-e610ec800434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.745845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa7019-3140-45cd-b4bf-e610ec800434 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.746112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa7019-3140-45cd-b4bf-e610ec800434 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.746380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efa7019-3140-45cd-b4bf-e610ec800434 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.747539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa7019-3140-45cd-b4bf-e610ec800434 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.749068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efa7019-3140-45cd-b4bf-e610ec800434 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.751010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 119/241] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:52.751967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.752248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.752519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.776385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:52.777563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:52.855130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7286015c-ee7d-43c5-b831-aefb2eb9983a None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:52.857699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 120/242] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:52.861968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:52.908943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:52.926146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:52.926862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:52.927143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:52.927416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:52.928092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:52.928396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:52.929807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:52.948061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 123/243] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:52.955188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.032798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001a3001-7920-4389-be06-dcf15d09c3eb None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.074558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.111659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.112268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.112535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.112800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.113450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.114904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.141637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807d04e7-c16f-46e0-a514-63482613e1fa None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.148902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 121/244] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:52 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:53.195340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.196000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.196276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.196543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] PATH_INFO: `/v3/users/c9ab8984dce14674b80f41d5a63a81a8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.207126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.234331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] RBAC: Authorizing `identity:get_user(user_id=c9ab8984dce14674b80f41d5a63a81a8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.235809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.237891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 124/245] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:52 2022] GET /identity/v3/users/c9ab8984dce14674b80f41d5a63a81a8 => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:53.243528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.382753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.398688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1edab6-1ab6-4175-824c-0a73fbdee19a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.401962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bb9754-72af-454b-ad44-82cb5b99fdb9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.427668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.443066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb161dc-ec62-4845-b7f6-646cfe238493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.444948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb161dc-ec62-4845-b7f6-646cfe238493 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.445546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb161dc-ec62-4845-b7f6-646cfe238493 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.446176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb161dc-ec62-4845-b7f6-646cfe238493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.448368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.448368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.448603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.448892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.462877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:53.468543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:53.469442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:53.474571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 122/246] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:53.484331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.536281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb161dc-ec62-4845-b7f6-646cfe238493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.538719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb161dc-ec62-4845-b7f6-646cfe238493 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.627095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787f523-ef03-41a2-8b53-f0d3028e7226 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.656639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 125/247] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:53.658458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998114f-f84e-44e7-8789-f3e9cc075c9e None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.659030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998114f-f84e-44e7-8789-f3e9cc075c9e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.659763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d998114f-f84e-44e7-8789-f3e9cc075c9e None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.662327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 126/248] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:53.663480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998114f-f84e-44e7-8789-f3e9cc075c9e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.678514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.679140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.679420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.679700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.680486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:53.680765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.683365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5839d47-2b72-4621-8c1d-582faefbbe2a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.692887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 123/249] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:53.694544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.694544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.694736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.699050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.718498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:53.719694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:53.721893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:53.798585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d998114f-f84e-44e7-8789-f3e9cc075c9e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:53.802859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 124/250] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:53.804344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:53.844136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e682b5d5-18fa-459d-ab56-e3dfba5bedcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:53.845669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b5d5-18fa-459d-ab56-e3dfba5bedcc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.846307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b5d5-18fa-459d-ab56-e3dfba5bedcc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.846820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e682b5d5-18fa-459d-ab56-e3dfba5bedcc None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.848133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682b5d5-18fa-459d-ab56-e3dfba5bedcc None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:53.850086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e682b5d5-18fa-459d-ab56-e3dfba5bedcc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:53.852865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 127/251] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:53.854223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adeaf-8eba-4ce5-afdd-1abe43b3aa67 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:53.854700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adeaf-8eba-4ce5-afdd-1abe43b3aa67 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:53.855158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2adeaf-8eba-4ce5-afdd-1abe43b3aa67 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:53.861774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 128/252] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:53.862818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2adeaf-8eba-4ce5-afdd-1abe43b3aa67 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.001888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.011782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73519368-c5f9-40c6-859f-e8c2fc603ccd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.013073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2adeaf-8eba-4ce5-afdd-1abe43b3aa67 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.053568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.054213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.054488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.054757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.055483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] RBAC: Adding query filter params (role.id=284c674552164365ab075b703005206e, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=c9ab8984dce14674b80f41d5a63a81a8) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:54.055868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.057338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8f063b-f3a4-4137-a9aa-d6c9a0d4ab22 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.064369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 129/253] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/role_assignments?user.id=c9ab8984dce14674b80f41d5a63a81a8&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=284c674552164365ab075b703005206e => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:54.066670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.067104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.067522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.087296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.088820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.089403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.089616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:54.090181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] PATH_INFO: `/v3/users/82e34f5c7ffc437fa88c6610b93cb150` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.091041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:54.092526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] RBAC: Authorizing `identity:get_user(user_id=82e34f5c7ffc437fa88c6610b93cb150)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.094290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.096178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 125/254] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:53 2022] GET /identity/v3/users/82e34f5c7ffc437fa88c6610b93cb150 => generated 297 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:54.107332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.120467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.178957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 130/255] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:54.179933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.324469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d924cde2-8694-4581-907b-7bba301bef87 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.326191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.353214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c046a34-8175-4ebc-93d9-78fa52f9dcd7 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.361547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.365752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.365752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.365752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.368211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.369648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.371518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 126/256] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:54.378337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.422586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.424373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.424657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.424932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.453193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:54.460732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:54.461677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:36:54.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:54.464598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 131/257] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:54.469351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.538076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.604683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9e3afa-e9cd-4b89-836d-6d7288130f87 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.665651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28fb2a-9bbe-4178-aa64-b52615781f4f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.683005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.683689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.683968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.684241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/82e34f5c7ffc437fa88c6610b93cb150/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.690562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] RBAC: Authorizing `identity:create_grant(project_id=69a6097d627b4c9da43f6fb3ca86f1ca, user_id=82e34f5c7ffc437fa88c6610b93cb150, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.696196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.724035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 127/258] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:54 2022] PUT /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/82e34f5c7ffc437fa88c6610b93cb150/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:54.725147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.747985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:54.748609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:54.748884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:54.749158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:54.749863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:54.750338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:54.751717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:54.775381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 132/259] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:54.780006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:54.798921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:54.960529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e10a79-61b7-4efc-9c36-1779f3fafa4b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:54.995829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aab3be-90b3-40ef-91ad-4a7223207239 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.033479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.034130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.034401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.034670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] PATH_INFO: `/v3/users/3d53df0fb9344a9ba064e57d504b6b44` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.035812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] RBAC: Authorizing `identity:get_user(user_id=3d53df0fb9344a9ba064e57d504b6b44)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.036244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.036953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.037237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.037338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.037516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.039200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 133/260] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/users/3d53df0fb9344a9ba064e57d504b6b44 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:55.039563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.046986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.066638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:55.072404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:55.073295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:36:55.074124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 128/261] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:36:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:55.085134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.179158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.210641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc98ad-054d-4c7e-861b-8f3d59c7d422 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.228217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f876123c-38fb-4168-857c-4bc425d5e236 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.264711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.265338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.265613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.265907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.266597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:55.266897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.268313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.270768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.271406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.271676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.271947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.298094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 129/262] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:55.306137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.390084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.397266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.398920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.459057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22cd444-0893-4dda-8ee6-d2c677f938eb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.513787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.516863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.517510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.518616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.519141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] PATH_INFO: `/v3/users/3d53df0fb9344a9ba064e57d504b6b44` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.521953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] RBAC: Authorizing `identity:get_user(user_id=3d53df0fb9344a9ba064e57d504b6b44)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.525011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.526640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 130/263] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/users/3d53df0fb9344a9ba064e57d504b6b44 => generated 283 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.533595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.559434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 134/264] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:55.566046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.711682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb921dc-1adf-4a5f-9e50-636892242f2f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.770174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:55.775370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02ee0c9-30eb-4388-b360-e945866e7458 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:55.802528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.803134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.803405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.803670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.862037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:55.862675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:55.862949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:55.863213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:55.865383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.866890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.868524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 135/265] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:55.874831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:55.935668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:55.937118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:55.966084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.037620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 131/266] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:36:56.043695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:56.068769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.107491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b10eea-9d24-4a90-b6fb-8bb19339d734 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.164580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d24b9a-19ca-46b5-bc1e-421189d2f5b2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.187332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.187956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.188238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.188511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.189243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=3d53df0fb9344a9ba064e57d504b6b44) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:56.189552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.191506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.204342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd643f6b-f105-4535-8056-562315bf599c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.204829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 136/267] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:55 2022] GET /identity/v3/role_assignments?user.id=3d53df0fb9344a9ba064e57d504b6b44&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:56.205588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643f6b-f105-4535-8056-562315bf599c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.206207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643f6b-f105-4535-8056-562315bf599c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.206672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd643f6b-f105-4535-8056-562315bf599c None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.208000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd643f6b-f105-4535-8056-562315bf599c None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.209660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd643f6b-f105-4535-8056-562315bf599c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.214842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 132/268] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:56 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:56.218671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:56.352095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc366cc-51f0-46f9-acb1-8356caf6717b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:56.363038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:56.391126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:56.392161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.392645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.393120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.394113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] RBAC: Adding query filter params (role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=3d53df0fb9344a9ba064e57d504b6b44) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:56.394627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:56.396300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32c9e25-bd10-44ca-b27b-1c92f8ea7e79 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:56.409089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 137/269] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:56 2022] GET /identity/v3/role_assignments?user.id=3d53df0fb9344a9ba064e57d504b6b44&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:56.483063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cd107-4478-4aac-a280-29cc1edcb41a None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.483714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cd107-4478-4aac-a280-29cc1edcb41a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.484184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7cd107-4478-4aac-a280-29cc1edcb41a None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.485793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 133/270] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:56.491692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e36aa-c62c-4077-b0f6-c3f904d51e0b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:56.492300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e36aa-c62c-4077-b0f6-c3f904d51e0b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:56.492802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e36aa-c62c-4077-b0f6-c3f904d51e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:56.515842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-290e36aa-c62c-4077-b0f6-c3f904d51e0b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:56.517547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-290e36aa-c62c-4077-b0f6-c3f904d51e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:56.620287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 138/271] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:56.944570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cd107-4478-4aac-a280-29cc1edcb41a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.051547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.055193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bb722-b16c-4c5b-ae38-6054a0068b9e None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.055511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bb722-b16c-4c5b-ae38-6054a0068b9e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.055737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76bb722-b16c-4c5b-ae38-6054a0068b9e None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.057089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 139/272] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:57.063240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf2e4e-6d8a-4302-84f2-741e33c91538 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf2e4e-6d8a-4302-84f2-741e33c91538 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.063741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaf2e4e-6d8a-4302-84f2-741e33c91538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.065445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7cd107-4478-4aac-a280-29cc1edcb41a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.098782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-abaf2e4e-6d8a-4302-84f2-741e33c91538 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:57.100378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-abaf2e4e-6d8a-4302-84f2-741e33c91538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:57.127157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.127793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.128065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.128327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.129438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.130910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:57.132796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 134/273] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:57.138601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.181251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 140/274] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:36:57.182552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09a35-e238-4cfa-b6fe-55b192d58121 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.182832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09a35-e238-4cfa-b6fe-55b192d58121 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.183103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d09a35-e238-4cfa-b6fe-55b192d58121 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.184357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 141/275] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:36:57.195386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.195858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.196131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.222933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:57.225062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:57.259639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.268287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff202918-bb80-470d-b6b8-cd1e5f480ade None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.303218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 142/276] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:57.310996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c1dc33-b234-4b8d-912e-a41001c2823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.311601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1dc33-b234-4b8d-912e-a41001c2823e None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.311876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1dc33-b234-4b8d-912e-a41001c2823e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.312145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c1dc33-b234-4b8d-912e-a41001c2823e None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.312801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1dc33-b234-4b8d-912e-a41001c2823e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.314270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c1dc33-b234-4b8d-912e-a41001c2823e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:57.339148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 135/277] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:36:57 2022] POST /identity/v3/users => generated 293 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:36:57.479511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.593329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f454fab2-a6cb-47d9-b5bb-ca8f15c97de3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.599833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d2c6-6574-41c4-89af-609a7fe3c7c7 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.600429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d2c6-6574-41c4-89af-609a7fe3c7c7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.600703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08d2c6-6574-41c4-89af-609a7fe3c7c7 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.602288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 136/278] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:36:57.607611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.607896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.608176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.629479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:36:57.630680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:36:57.637652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.638624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.638889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.639163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.645089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.654480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:57.659623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:57.660517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:57.661290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 143/279] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:57.662060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.691720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.728236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 137/280] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:36:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:36:57.730053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.793232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24f3b8-0ba4-4b38-96df-8a8c90818b60 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.840082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.840991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.841268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.841536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.852952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:57.860320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:57.861211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:57.862105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 144/281] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:57.874193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:57.875968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dc4267-bea4-4727-8ef6-5bfd93bd3f3e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:57.977072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:57.979457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:57.980183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:57.980613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:57.980966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:57.984374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:57.984678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:57.986082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.004403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 138/282] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.010437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021b853-2f9e-403b-a662-f30262072438 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.018935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.034023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.078521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.079316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.079596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.079865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.080691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] RBAC: Adding query filter params (name=reader) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:58.081528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.083366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.095596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 145/283] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:36:58.097083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.145998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb331074-b930-4d16-aa6c-744cb9ca5969 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.187918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341165-0141-4957-a0a1-24842e615040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.188799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341165-0141-4957-a0a1-24842e615040 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.189224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341165-0141-4957-a0a1-24842e615040 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.189664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79341165-0141-4957-a0a1-24842e615040 None admin] PATH_INFO: `/v3/users/7b697434916b4c97bb8d279a38623e3e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.208458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79341165-0141-4957-a0a1-24842e615040 None admin] RBAC: Authorizing `identity:get_user(user_id=7b697434916b4c97bb8d279a38623e3e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.210929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79341165-0141-4957-a0a1-24842e615040 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.213842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 139/284] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/users/7b697434916b4c97bb8d279a38623e3e => generated 293 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.215002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341165-0141-4957-a0a1-24842e615040 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.219211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62088fbd-717a-40d8-a654-8ebe56305aad None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.263189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.264088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.264512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.264926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.278309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:36:58.285381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:36:58.286461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:36:58.287450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:36:58.289803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 146/285] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:36:58.291139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.295799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.297108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.348038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79341165-0141-4957-a0a1-24842e615040 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.385480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.386982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.387457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.387969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] PATH_INFO: `/v3/users/c9ab8984dce14674b80f41d5a63a81a8` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.389346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] RBAC: Authorizing `identity:get_user(user_id=c9ab8984dce14674b80f41d5a63a81a8)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.391121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.393017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 140/286] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/users/c9ab8984dce14674b80f41d5a63a81a8 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.394181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.411943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1960f5ef-798f-4028-aac7-47260aca1cd4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.451742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.452692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.453117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.453537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.454750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, user.id=7b697434916b4c97bb8d279a38623e3e, scope.system=all) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:58.455414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.457692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.464378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 147/287] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/role_assignments?user.id=7b697434916b4c97bb8d279a38623e3e&scope.system=all&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.465485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.508097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6c35-0b3e-41ab-aba6-ad34b42d1eaf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.543414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.556504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.557409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.558486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.558994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.559876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:58.560372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.562167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.564256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.574204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 141/288] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.587031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.587862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c554b8f-ed57-47c3-90f7-2463be407882 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.626382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.627596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.628104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.628598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.630077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.631824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.634092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 148/289] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.639890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.736837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ebb589-c89c-4191-b08c-c79d99526a9c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.787385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.788696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.789506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.790030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] PATH_INFO: `/v3/users/82e34f5c7ffc437fa88c6610b93cb150` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.791160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccb1d7d-2a8f-4ee0-ada8-136e4ed49640 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.792213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] RBAC: Authorizing `identity:get_user(user_id=82e34f5c7ffc437fa88c6610b93cb150)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.795074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.797406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 142/290] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/users/82e34f5c7ffc437fa88c6610b93cb150 => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:58.802548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:58.882983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.883660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.883933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.884204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/c9ab8984dce14674b80f41d5a63a81a8/roles/284c674552164365ab075b703005206e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:58.891087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] RBAC: Authorizing `identity:create_grant(project_id=69a6097d627b4c9da43f6fb3ca86f1ca, user_id=c9ab8984dce14674b80f41d5a63a81a8, role_id=284c674552164365ab075b703005206e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:58.892241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.892459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:58.902915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:58.937401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 149/291] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:36:58 2022] PUT /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/users/c9ab8984dce14674b80f41d5a63a81a8/roles/284c674552164365ab075b703005206e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:36:58.939453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62abf64-85cc-4323-981e-b2d5d835bc26 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:58.980006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:58.981026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:58.981504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:58.982008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.060234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.062042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.129791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:36:59.174356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 143/292] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:36:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:36:59.181768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.315088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd597bb5-a35f-4218-87b4-9a437a71466c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.358213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8d6253-281f-4792-bea0-de2fab5a8b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.358852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d6253-281f-4792-bea0-de2fab5a8b78 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.359126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d6253-281f-4792-bea0-de2fab5a8b78 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.359393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8d6253-281f-4792-bea0-de2fab5a8b78 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.360516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d6253-281f-4792-bea0-de2fab5a8b78 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.361967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8d6253-281f-4792-bea0-de2fab5a8b78 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.364571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 150/293] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:36:59.369384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:36:59.498790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3b914f-75af-4357-b3d0-a80851416886 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:36:59.547664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:36:59.548679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:36:59.549224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:36:59.549692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:36:59.550681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=82e34f5c7ffc437fa88c6610b93cb150) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:36:59.551221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:36:59.552904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130d7e90-9a63-4851-95bf-b89ef15aa374 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:36:59.560907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 144/294] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:36:59 2022] GET /identity/v3/role_assignments?user.id=82e34f5c7ffc437fa88c6610b93cb150&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.024472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c9e5-ca8e-42a4-b6d1-c97ca66dcfa8 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.024855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c9e5-ca8e-42a4-b6d1-c97ca66dcfa8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.025031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28c9e5-ca8e-42a4-b6d1-c97ca66dcfa8 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.026455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 151/295] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:00.032149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9f7b0-4112-46e1-9e5a-73a5010ea6fb None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.032453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9f7b0-4112-46e1-9e5a-73a5010ea6fb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.032710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c9f7b0-4112-46e1-9e5a-73a5010ea6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.053387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-11c9f7b0-4112-46e1-9e5a-73a5010ea6fb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:00.054705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-11c9f7b0-4112-46e1-9e5a-73a5010ea6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:00.081646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.136211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 145/296] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:00.164047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d475f-f878-44a9-a305-7d0ed193aedd None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.164360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d475f-f878-44a9-a305-7d0ed193aedd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.164606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561d475f-f878-44a9-a305-7d0ed193aedd None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.166834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 152/297] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:00.171795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.172082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.172355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.191410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:00.192600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:00.272350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 146/298] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:00.416631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d475f-f878-44a9-a305-7d0ed193aedd None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.484231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.524556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561d475f-f878-44a9-a305-7d0ed193aedd None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.557126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.561203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.561889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.562169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.562443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.576470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:00.583744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:00.584644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:00.585423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:00.586063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 153/299] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:00.592481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.660511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.675356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fcb0a9-6694-427a-93cf-b61c1606479a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.714419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.715260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.715697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.716102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.717329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.718942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.720737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 147/300] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:00.726930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.727967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b164b76-4e44-4296-a1b9-7079589c59d5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.756877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:00.770388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.771268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.771834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.772296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.773152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:00.773638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.775762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.789182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 154/301] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.790404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:00.844695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4a936e-701a-4578-bb99-c86a7332cbee None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.888581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.889206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.889478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.889759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] PATH_INFO: `/v3/groups` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.890429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.892263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.912528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857159c9-70ba-4edd-a3d3-8759f0f65927 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:00.914724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 148/302] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Sep 13 19:37:00 2022] POST /identity/v3/groups => generated 229 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:00.958619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc1b70-cb74-4b0d-be3d-6573079bf66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:00.959691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc1b70-cb74-4b0d-be3d-6573079bf66a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:00.960211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc1b70-cb74-4b0d-be3d-6573079bf66a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:00.960782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc1b70-cb74-4b0d-be3d-6573079bf66a None admin] PATH_INFO: `/v3/users/691f69b2cb5f4a2f80f69b6b2f241155` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:00.978707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc1b70-cb74-4b0d-be3d-6573079bf66a None admin] RBAC: Authorizing `identity:get_user(user_id=691f69b2cb5f4a2f80f69b6b2f241155)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:00.980579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffc1b70-cb74-4b0d-be3d-6573079bf66a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:00.982998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 155/303] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/users/691f69b2cb5f4a2f80f69b6b2f241155 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:00.989116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.000803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e527-ae99-4817-b884-930282970041 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.001415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e527-ae99-4817-b884-930282970041 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.001996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3511e527-ae99-4817-b884-930282970041 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.003873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 156/304] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:01.010803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.011453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.011941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.015230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.036006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:01.037582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:01.094322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.110051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5291e6-f4d1-4cd4-9891-c8b666f3f042 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.127075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 157/305] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:01.153410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.154098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.154378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.154645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.155360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] RBAC: Adding query filter params (role.id=284c674552164365ab075b703005206e, user.id=691f69b2cb5f4a2f80f69b6b2f241155, scope.system=all) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:01.155659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.157176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac847f45-28e3-4855-9578-00018e42752e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.164395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 149/306] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 19:37:00 2022] GET /identity/v3/role_assignments?user.id=691f69b2cb5f4a2f80f69b6b2f241155&scope.system=all&role.id=284c674552164365ab075b703005206e => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:01.395220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.448272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31106aa4-c62e-42e2-8d57-8dba6989730c None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.449038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31106aa4-c62e-42e2-8d57-8dba6989730c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.449406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31106aa4-c62e-42e2-8d57-8dba6989730c None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.451590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 150/307] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:01.457100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.457383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.457654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.478672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:01.480183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:01.513693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de10da06-0822-4606-9bff-24cd55dce276 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.565767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 151/308] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:01.568361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.569173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.569651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.570121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.580886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:01.585971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:01.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:01.588146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:01.590371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 158/309] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:01.597770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.610090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.700195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f03a5-a978-44cc-9fda-00206726d161 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.734642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.735258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.735532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.736190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.736880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:01.737183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.738637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.749416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 152/310] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:01.755036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.791969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.849841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.867173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac4d88-2469-4f9c-b97e-e784c7b87e6b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:01.904729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:01.905530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:01.905833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:01.906109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] PATH_INFO: `/v3/users/7b697434916b4c97bb8d279a38623e3e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:01.907258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] RBAC: Authorizing `identity:get_user(user_id=7b697434916b4c97bb8d279a38623e3e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:01.908929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:01.910852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 159/311] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/users/7b697434916b4c97bb8d279a38623e3e => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:01.917031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:01.932218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:01.976158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17d058-cab2-44df-a9c1-a2fe31121cf5 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.023534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.025286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.026071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.026931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.029761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e10538-4a97-4bab-b53e-02cd42361990 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.031546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.043276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:02.051508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:02.052426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:02.053222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:02.059496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 153/312] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:02.061056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.075851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.076279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.076558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.076834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] PATH_INFO: `/v3/system/users/7b697434916b4c97bb8d279a38623e3e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.078332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7b697434916b4c97bb8d279a38623e3e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.079782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.090660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 160/313] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Sep 13 19:37:01 2022] PUT /identity/v3/system/users/7b697434916b4c97bb8d279a38623e3e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:02.183036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4094095d-3a12-463a-a49b-6de809f1e5cc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.216744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.222488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.223479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.223958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.224424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.225317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] RBAC: Adding query filter params (name=reader) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:02.225873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.228140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.238745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 154/314] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.244902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.359745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc250421-8dcc-4ad7-ab5d-c5a540f7bc8b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.394804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.395840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.396317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.396785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] PATH_INFO: `/v3/users/c9ab8984dce14674b80f41d5a63a81a8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.398282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] RBAC: Authorizing `identity:get_user(user_id=c9ab8984dce14674b80f41d5a63a81a8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.400465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.402598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 161/315] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/users/c9ab8984dce14674b80f41d5a63a81a8 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.408313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.511859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4ef74-ea8f-402e-8488-89d7f905beda None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.547536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.548538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.549082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.549546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.611525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.619108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.620856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.727155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 155/316] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:02.732522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.770609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.842191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253faf7-7354-453d-b01b-cc445163d5df None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:02.877481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a9e61d-92df-49f2-aad7-d5c86898d9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:02.878152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9e61d-92df-49f2-aad7-d5c86898d9cf None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:02.878433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9e61d-92df-49f2-aad7-d5c86898d9cf None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:02.878705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9e61d-92df-49f2-aad7-d5c86898d9cf None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:02.879842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9e61d-92df-49f2-aad7-d5c86898d9cf None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:02.881280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a9e61d-92df-49f2-aad7-d5c86898d9cf None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:02.882944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 162/317] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:02.889533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:02.987095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:02.995891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235b2a1-a608-48aa-a3c6-4dd0b45276bf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.031967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.032897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.033393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.033889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.034814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] RBAC: Adding query filter params (role.id=284c674552164365ab075b703005206e, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca, user.id=c9ab8984dce14674b80f41d5a63a81a8) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:03.035399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.037024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4036bba4-d972-4cc3-afe0-8da07bc43788 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.044235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 156/318] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:02 2022] GET /identity/v3/role_assignments?user.id=c9ab8984dce14674b80f41d5a63a81a8&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=284c674552164365ab075b703005206e => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:03.248527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea31b7-5ac4-410a-b142-9d9f0fd903b2 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.248841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea31b7-5ac4-410a-b142-9d9f0fd903b2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.249189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ea31b7-5ac4-410a-b142-9d9f0fd903b2 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.250696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 163/319] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:03.256130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbbb0-4ffb-4b5e-a1c9-8be753b62bd6 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.256414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbbb0-4ffb-4b5e-a1c9-8be753b62bd6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.256817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74efbbb0-4ffb-4b5e-a1c9-8be753b62bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.275970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-74efbbb0-4ffb-4b5e-a1c9-8be753b62bd6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:03.277158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-74efbbb0-4ffb-4b5e-a1c9-8be753b62bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:03.348636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 157/320] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:03.535835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29be1c1-36c2-4342-aade-57cd12b141e3 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.536194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29be1c1-36c2-4342-aade-57cd12b141e3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.536398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29be1c1-36c2-4342-aade-57cd12b141e3 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.537919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 164/321] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:03.544517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.545056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.545056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.563225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:03.564401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:03.615272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29be1c1-36c2-4342-aade-57cd12b141e3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.632575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 158/322] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:03.739903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29be1c1-36c2-4342-aade-57cd12b141e3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.742443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.777626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.778283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.778559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.778829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.779919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.781357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.783114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 165/323] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:03.788519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.807721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:03.895905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06416bbb-0a72-49e1-979f-0dbea18cb492 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:03.906515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:03.930480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:03.931083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:03.931362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:03.931629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] PATH_INFO: `/v3/groups` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:03.932268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:03.933684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:03.961913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 159/324] 10.222.0.15 () {60 vars in 1134 bytes} [Tue Sep 13 19:37:03 2022] POST /identity/v3/groups => generated 226 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:04.023135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d4814-2a7f-43ad-9158-d25ce6e19218 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.060199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.060842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.061115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.061384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.073059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:04.077954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:04.078880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:04.084967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 166/325] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:04.085660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.168920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.193499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5458fe2c-1e6e-4850-92d5-02ea2d7e2b35 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.230518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.231083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.231352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.231663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.232279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:04.232571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.233944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281df001-a1e1-4c6e-b371-5340a30195a4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.244333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 160/326] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:04.250624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.292408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.365767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262af080-4cb8-4f40-bb64-712147d95594 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.375662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7e34-92e6-4896-b562-25b4b41582e2 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.375948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7e34-92e6-4896-b562-25b4b41582e2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.376207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a7e34-92e6-4896-b562-25b4b41582e2 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.377836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 161/327] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:04.383724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.384003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.384277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.403168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.404263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.404538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.404802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] PATH_INFO: `/v3/users/691f69b2cb5f4a2f80f69b6b2f241155` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] RBAC: Authorizing `identity:get_user(user_id=691f69b2cb5f4a2f80f69b6b2f241155)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.407998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.410165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 167/328] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/users/691f69b2cb5f4a2f80f69b6b2f241155 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:04.414729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:04.415759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.419157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:04.504355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 162/329] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:04.535503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.540901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabcb95-a6e8-4d6f-aa31-cc026877b508 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.579680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.580336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.580616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.580892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] PATH_INFO: `/v3/system/users/691f69b2cb5f4a2f80f69b6b2f241155/roles/284c674552164365ab075b703005206e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.582387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=691f69b2cb5f4a2f80f69b6b2f241155, role_id=284c674552164365ab075b703005206e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:04.583887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:04.593821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 168/330] 10.222.0.15 () {58 vars in 1334 bytes} [Tue Sep 13 19:37:04 2022] PUT /identity/v3/system/users/691f69b2cb5f4a2f80f69b6b2f241155/roles/284c674552164365ab075b703005206e => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:04.824943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:04.905109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:04.938052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a9112-0ecc-41b9-a0aa-6d93722a47c0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:04.982457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:04.983097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:04.983369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:04.983634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:04.997344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:05.002417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:05.003379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:05.004158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 163/331] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:05.009806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.172623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675e84ec-b78a-4cc3-b223-a0b87b7b4d59 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.218503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.219117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.219397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.219672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.220344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] RBAC: Adding query filter params (name=member) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:05.220648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.222049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.231497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 169/332] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:05.237170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.406337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdcedb-50ed-4ccc-bbfc-f4ef2925bffc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.448227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.454028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0420836-2190-46d4-989c-7aedee9fcd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.454673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0420836-2190-46d4-989c-7aedee9fcd1c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.455157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0420836-2190-46d4-989c-7aedee9fcd1c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.455427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0420836-2190-46d4-989c-7aedee9fcd1c None admin] PATH_INFO: `/v3/users/7b697434916b4c97bb8d279a38623e3e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.456716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0420836-2190-46d4-989c-7aedee9fcd1c None admin] RBAC: Authorizing `identity:get_user(user_id=7b697434916b4c97bb8d279a38623e3e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.458250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0420836-2190-46d4-989c-7aedee9fcd1c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.459832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 164/333] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/users/7b697434916b4c97bb8d279a38623e3e => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:05.466004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:05.516955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:05.615917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb001e-b1fb-4702-99a0-6fc2ba9b2a0e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:05.655670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:05.656287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:05.656567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:05.656837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:05.657566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] RBAC: Adding query filter params (role.id=61f630f0ba254ac5bb25aa078b9876a0, user.id=7b697434916b4c97bb8d279a38623e3e, scope.system=all) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:05.657899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:05.659315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82777a-c9c3-4aa8-a336-4c759a1a207e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:05.675588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 170/334] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 19:37:05 2022] GET /identity/v3/role_assignments?user.id=7b697434916b4c97bb8d279a38623e3e&scope.system=all&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:06.739880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2cb55-51fe-4599-9840-488efe7b3dce None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.740905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2cb55-51fe-4599-9840-488efe7b3dce None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.741968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2cb55-51fe-4599-9840-488efe7b3dce None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.743756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 165/335] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:06.749894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e6a5c-2d11-4761-870a-18f574c48a5b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:06.750822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e6a5c-2d11-4761-870a-18f574c48a5b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:06.751289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8e6a5c-2d11-4761-870a-18f574c48a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:06.773793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bf8e6a5c-2d11-4761-870a-18f574c48a5b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:06.775325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bf8e6a5c-2d11-4761-870a-18f574c48a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:06.863466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 171/336] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:07.025254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e275-5ebe-416d-87ed-f8503cb29850 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.025610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e275-5ebe-416d-87ed-f8503cb29850 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.026106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e94e275-5ebe-416d-87ed-f8503cb29850 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.027780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 166/337] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:07.033491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.033798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.034065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.062909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:07.064123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:07.073188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.167346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 172/338] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:07.259784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e275-5ebe-416d-87ed-f8503cb29850 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.382768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e94e275-5ebe-416d-87ed-f8503cb29850 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.428097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.428760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.429032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.429294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.442531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:07.447409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:07.448277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:07.449696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 167/339] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:07.454364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.567415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85df5c-99eb-4ac3-9999-7e538ff66942 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.611560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.612169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.612440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.612710 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.613371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:07.613669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.615082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.627879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 173/340] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:07.634212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.700180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:07.752503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab222aa-ae3b-43cb-9b03-cc721021f90c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:07.797948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f1873-783d-463e-b341-6974a40c2bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:07.798557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1873-783d-463e-b341-6974a40c2bf2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:07.798830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1873-783d-463e-b341-6974a40c2bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:07.799093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f1873-783d-463e-b341-6974a40c2bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:07.890006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f1873-783d-463e-b341-6974a40c2bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:07.891379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f1873-783d-463e-b341-6974a40c2bf2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:07.898788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:07.919992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.022119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.118193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de4380-bab8-4291-ae15-0fc4d51403cc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.126610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 168/341] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:08.127865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506013e-2d69-4033-938b-95f2ed14f895 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.128143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506013e-2d69-4033-938b-95f2ed14f895 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.128409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506013e-2d69-4033-938b-95f2ed14f895 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.129902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 169/342] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:08.139257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.139521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.139788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.165418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.166198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.166474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.166743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.172525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:08.173714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:08.202457 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:08.208064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:08.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:08.210241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 174/343] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:08.214927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.272443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 170/344] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:08.273323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.370636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.380576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.396654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9044c6-d2d3-46a1-909d-6d7c6387ad3e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.440768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.441305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.441633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.441934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.442614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:08.442913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.444326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e071c7-525d-4e81-b571-45fde8f4a82b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.462932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c838d02-0a92-479d-951e-c4a6bbcb6590 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.465752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 171/345] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.466995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062f5b-f2f0-447a-a74c-eef920e05d57 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.467275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062f5b-f2f0-447a-a74c-eef920e05d57 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.467546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8062f5b-f2f0-447a-a74c-eef920e05d57 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.468745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 172/346] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:08.476321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99deae-d918-4311-8e70-996c76304226 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.476593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99deae-d918-4311-8e70-996c76304226 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.477098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99deae-d918-4311-8e70-996c76304226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.499152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-fc99deae-d918-4311-8e70-996c76304226 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:08.500720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-fc99deae-d918-4311-8e70-996c76304226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:08.555989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.557033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.557516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.558035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.559428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.561132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.570086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 175/347] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.571430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.625309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 173/348] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:08.626649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99deae-d918-4311-8e70-996c76304226 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.769444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b8f390-9fbe-44e3-953c-467f3fc33ea0 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.792395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.806976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:08.812438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99deae-d918-4311-8e70-996c76304226 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:08.812852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaacd569-ec41-4a9f-980e-8b695bddfe86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.813535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd569-ec41-4a9f-980e-8b695bddfe86 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.815066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd569-ec41-4a9f-980e-8b695bddfe86 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.815368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaacd569-ec41-4a9f-980e-8b695bddfe86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.851713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e771-5536-4354-abae-314813d0f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:08.852348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e771-5536-4354-abae-314813d0f389 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:08.852620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e771-5536-4354-abae-314813d0f389 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:08.852889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e771-5536-4354-abae-314813d0f389 None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:08.859030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073e771-5536-4354-abae-314813d0f389 None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.860459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a073e771-5536-4354-abae-314813d0f389 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.864239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 174/349] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:08.865030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e771-5536-4354-abae-314813d0f389 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:08.896858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd569-ec41-4a9f-980e-8b695bddfe86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:08.898419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaacd569-ec41-4a9f-980e-8b695bddfe86 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:08.979154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e771-5536-4354-abae-314813d0f389 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.013553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 176/350] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:09.018144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.018144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.018144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.018144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.019942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab446e-8bd3-4ee1-9af3-807e09fd0ada None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.020381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab446e-8bd3-4ee1-9af3-807e09fd0ada None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.020806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeab446e-8bd3-4ee1-9af3-807e09fd0ada None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.023146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 177/351] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:09.024125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab446e-8bd3-4ee1-9af3-807e09fd0ada None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.036702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:09.042664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:09.043942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 19:37:09.044801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:09.046956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 175/352] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:09.047925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.064321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.065561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.153555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeab446e-8bd3-4ee1-9af3-807e09fd0ada None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.197494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.198141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.198419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.198682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.198783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ff6bc-137d-415d-8f34-a45584916492 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.199417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=22554735f59a426ea67d528448c2727d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.199721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.203204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.210472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 178/353] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.211035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.242608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b573007-6d6e-4844-8880-9708ca1e0759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.243690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b573007-6d6e-4844-8880-9708ca1e0759 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.244060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b573007-6d6e-4844-8880-9708ca1e0759 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.244422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b573007-6d6e-4844-8880-9708ca1e0759 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.259887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b573007-6d6e-4844-8880-9708ca1e0759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3b573007-6d6e-4844-8880-9708ca1e0759 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:09.265590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:09.266616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:09.267418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:09.268698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 176/354] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:09.270074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.270361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.270806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.292535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:09.294507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:09.360296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.374315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5c8b-9def-46ef-9a2c-f702389a1011 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.381339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.392460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 177/355] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:09.393390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.435017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.436046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.436550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.437022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.441847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.444351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.446786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 179/356] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.447785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.555764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc74f9-1055-44f0-b87f-d6e2ea54b45a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.615963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.616937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.617584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.618439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.619633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] RBAC: Adding query filter params (name=reader) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.620303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.622559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.627836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c619f7-eeba-4dbf-9d79-ac4b1f6c7818 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.634644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 178/357] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.636259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.663365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.663976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.664315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.664643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.665707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] RBAC: Adding query filter params (name=member) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:09.666108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.668677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96adc3-52b9-4c13-b8f5-7e67e6888b92 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.679134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 180/358] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.680197 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275310e6-716b-4c0d-bacb-740453112ff1 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.680479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275310e6-716b-4c0d-bacb-740453112ff1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.680748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275310e6-716b-4c0d-bacb-740453112ff1 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.686424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 181/359] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:09.687052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275310e6-716b-4c0d-bacb-740453112ff1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.721553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.728610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.772381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d0dd34-8069-4ccb-8ea6-1db43ea6f55a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.814404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.815266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.815496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.815803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.816873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.817241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275310e6-716b-4c0d-bacb-740453112ff1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.818396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.820371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 179/360] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.820821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.857778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30867c43-4181-475d-ac68-7811729a9447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.858382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30867c43-4181-475d-ac68-7811729a9447 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.858660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30867c43-4181-475d-ac68-7811729a9447 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.858929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30867c43-4181-475d-ac68-7811729a9447 None admin] PATH_INFO: `/v3/users/691f69b2cb5f4a2f80f69b6b2f241155` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.860050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30867c43-4181-475d-ac68-7811729a9447 None admin] RBAC: Authorizing `identity:get_user(user_id=691f69b2cb5f4a2f80f69b6b2f241155)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:09.861502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30867c43-4181-475d-ac68-7811729a9447 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:09.863818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 182/361] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/users/691f69b2cb5f4a2f80f69b6b2f241155 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:09.865824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.865824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.866942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:09.887081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:09.888372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:09.942410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3457a4b-5d5e-4c2e-a8e5-e705fa94c5da None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:09.969236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 183/362] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:09.971082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:09.974640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:09.987392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:09.988413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:09.988897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:09.989265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.003836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.083567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.085052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.094780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b411e3-edca-4664-950e-3f330b60d81f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.132947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.133976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.134553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.135032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.147608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:10.153355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.154549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:10.155353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:10.157686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 184/363] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:10.158705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.193847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 180/364] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:10.194681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.227124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.268500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.290812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da8da8-31b3-43e7-8bec-711084050551 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.320306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478a6ffb-f1a4-4354-b9f5-c07f76d8deaa None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.330348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.331556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.332053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.332793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.334638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=22554735f59a426ea67d528448c2727d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.335140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.337430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.347401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 185/365] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.348754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.366394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.367014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.367581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.367853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.369075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] RBAC: Adding query filter params (role.id=284c674552164365ab075b703005206e, user.id=691f69b2cb5f4a2f80f69b6b2f241155, scope.system=all) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.369505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.371289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.388430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 181/366] 10.222.0.15 () {56 vars in 1308 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/role_assignments?user.id=691f69b2cb5f4a2f80f69b6b2f241155&scope.system=all&role.id=284c674552164365ab075b703005206e => generated 510 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.389448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.545538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151c631-02ba-46b4-86fd-5e4f13471f9f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.556643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.579586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d831d2-6a54-42c6-b540-7f4a52ee7a0f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.604432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.604991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.605269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.605536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.606282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:10.606582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.608018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.624540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 186/367] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.625423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.639837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.640538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.640808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.641524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.642945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.643876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.646524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.649385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 182/368] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:10.650761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.759942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b03054-a6c5-4f80-a954-d94c49519753 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.776831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bfe233-c4ee-4be8-b94e-fd46a0d238bd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:10.802270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.803384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.803730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.804071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.818664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:10.820812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:10.822036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:10.822488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:10.822930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:10.826462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:10.827359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:10.828152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:10.830878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 187/369] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:10.837261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:10.855052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.911807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:10.913858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:10.928255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:10.965358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4df0fd-270b-4491-8670-dd6ef277866c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.006435 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.007107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.007382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.007651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.009051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.011244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.013112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 188/370] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.014163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.036847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 183/371] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:11.037792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.105139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.143267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fc92f-5365-4a80-9b9c-a805ecfb989f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.169669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2cf528-f7fb-4c7d-83ca-3f59ba2c6047 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.199262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.199887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.200167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.200440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.201665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.202051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.203722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.211767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.217440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 189/372] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.218947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.220639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.221479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.221984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.222429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.223360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.223913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.225572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.232906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 184/373] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.233936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.394552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4ecbeb-6cfb-4d5a-b6de-22b6e66f75e5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.405920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a23d7-1172-4eb2-9580-1a31fd50c6bd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.423951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.458956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.459886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.460326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.460750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.484853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.485860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.487981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.488630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.490739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.493556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.496072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 190/374] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.502187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.618557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.620034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.645972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.705759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7de227-909e-43f8-a0f4-6c67c84edae6 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.758836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.760008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.761380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.762846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.765285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.768076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.775956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 191/375] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.783346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.803320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 185/376] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:11.809000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:11.832069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.919282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7f5964-d4d8-4655-802d-00d7ccb43803 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.950027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1c435-8d43-44bc-9192-706ecd258d4b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:11.965113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.966107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.966591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.967048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.967942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:11.968424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.970027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115253f3-acaf-405a-a104-9ab00e3156ea None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:11.973375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:11.977911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 192/377] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:11.979496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a69a-97da-442b-b1b9-01f3f3ee50ef None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.980007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a69a-97da-442b-b1b9-01f3f3ee50ef None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.980380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a69a-97da-442b-b1b9-01f3f3ee50ef None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.981865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 193/378] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:11.987584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.987983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.988340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.991460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:11.992595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:11.993029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:11.993441 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] PATH_INFO: `/v3/projects/35cfa2e060644668a62b0b52ed4b9871` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:11.994940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] RBAC: Authorizing `identity:get_project(project_id=35cfa2e060644668a62b0b52ed4b9871)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:11.996770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.001222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 186/379] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:11 2022] GET /identity/v3/projects/35cfa2e060644668a62b0b52ed4b9871 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:12.007263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.009188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:12.010513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:12.162593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 194/380] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:12.182255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.203305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e83bf4e-c10b-4486-ad22-424bb0bf979d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.245826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.246453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.246727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.246993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] PATH_INFO: `/v3/groups/6ff62cc132a04937af9be06e4a61b954` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.252671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] RBAC: Authorizing `identity:get_group(group_id=6ff62cc132a04937af9be06e4a61b954)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.254174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.256763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 187/381] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/groups/6ff62cc132a04937af9be06e4a61b954 => generated 229 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:12.262224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.371934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3af20-64c8-4363-835b-9949133a49f8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.405438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:12.413088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.414228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.414893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.415540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.416263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] RBAC: Adding query filter params (group.id=6ff62cc132a04937af9be06e4a61b954, role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=35cfa2e060644668a62b0b52ed4b9871) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:12.416562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:12.418040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:12.424683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 195/382] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/role_assignments?group.id=6ff62cc132a04937af9be06e4a61b954&scope.project.id=35cfa2e060644668a62b0b52ed4b9871&role.id=40a9647e8ec745c999c2457d053557a7 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:12.645712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:12.772547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e9bb2-4927-489f-abfc-c9b587d700f5 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:12.857235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:12.857915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:12.858159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:12.858425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:12.875897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:12.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:12.881615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:12.882452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 188/383] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:12.888012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.034111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa1b44-a75d-44ba-a20c-f60a6a1810bd None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.075477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.076092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.076370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.076642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.077359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:13.077670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.079108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.091974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 196/384] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:13.097639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.111462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.245497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a89431-86c5-4743-8737-f3870fbe5179 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.289387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b7827e-4661-4816-b0da-b34c56fd93ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.289959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7827e-4661-4816-b0da-b34c56fd93ef None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.290232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7827e-4661-4816-b0da-b34c56fd93ef None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.290559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b7827e-4661-4816-b0da-b34c56fd93ef None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.291681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7827e-4661-4816-b0da-b34c56fd93ef None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.293094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b7827e-4661-4816-b0da-b34c56fd93ef None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.294981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 189/385] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:13.300303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.310150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0746-f3ea-40e2-9ea2-7be1f6c24d34 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.310427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0746-f3ea-40e2-9ea2-7be1f6c24d34 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.310759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0746-f3ea-40e2-9ea2-7be1f6c24d34 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.312322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 190/386] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:13.323757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.324053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.324323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.362809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:13.364086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:13.425512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.509062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 191/387] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:13.521107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9d91d-d010-467a-9b05-88816cb2fd1a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.564815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.565371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.565652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.565959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.569666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.571144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.573106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 197/388] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:13.578665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.583130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:13.710338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289f7196-01b9-4a30-9b49-8efc5baf6ab2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:13.749125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9155b-46d1-422c-ba10-6874d180f5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:13.749812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9155b-46d1-422c-ba10-6874d180f5e7 None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:13.750094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9155b-46d1-422c-ba10-6874d180f5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:13.750366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9155b-46d1-422c-ba10-6874d180f5e7 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:13.752550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9155b-46d1-422c-ba10-6874d180f5e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=22554735f59a426ea67d528448c2727d, group_id=90bb2e95f23b4bdb8ecebfc098856afe, role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:13.754522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9155b-46d1-422c-ba10-6874d180f5e7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:13.778408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 192/389] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 19:37:13 2022] PUT /identity/v3/projects/22554735f59a426ea67d528448c2727d/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:13.854925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:13.986053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a1e0a9-ab94-43d9-b5ae-4c7e6495ab3e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.016850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.028590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.029228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.029509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.029803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.036076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c77a7f-d7dc-4b25-9abc-296fed803d9d None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.036580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c77a7f-d7dc-4b25-9abc-296fed803d9d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.036580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c77a7f-d7dc-4b25-9abc-296fed803d9d None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.037965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 193/390] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:14.043613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.043932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.044179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.054604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:14.061587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:14.062576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:14.063613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:14.066011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 198/391] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:14.071874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.077765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:14.078989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:14.180925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.196722 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 194/392] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:14.213410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087948d6-73b3-4206-9eb8-4f410f56f130 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.250331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.250943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.251224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.251504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.252181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] RBAC: Adding query filter params (name=member) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:14.252489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.254527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.263538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 199/393] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:14.269064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.377359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c776e-1355-44c8-b813-3bcfd1a44e2b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.413205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.413867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.414145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.414414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.415521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.416934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.418744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 195/394] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:14.424258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.480625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.493496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.584652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be8d62-d712-4099-83e5-de95c30cbe4e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.608171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.660851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.661495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.661796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.662074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.667182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.668639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.673570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 200/395] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:14.688343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40edf576-1c75-4d8e-bae0-062e2c506d0b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.708256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.762817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.764842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.765129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.765402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.781711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:14.787727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:14.788685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:14.790114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 196/396] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:14.797721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:14.877228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60fd3bd-0f96-4184-a075-19b2785aaa0f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.894804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.921718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff993b-4939-43c2-82c1-18c2436f198e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.923206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff993b-4939-43c2-82c1-18c2436f198e None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.923955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff993b-4939-43c2-82c1-18c2436f198e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.924455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff993b-4939-43c2-82c1-18c2436f198e None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.927858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff993b-4939-43c2-82c1-18c2436f198e None admin] RBAC: Authorizing `identity:create_grant(project_id=22554735f59a426ea67d528448c2727d, group_id=90bb2e95f23b4bdb8ecebfc098856afe, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:14.930131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff993b-4939-43c2-82c1-18c2436f198e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:14.943402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4bd44-6b04-4325-a979-8c45837cb775 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:14.949108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 201/397] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 19:37:14 2022] PUT /identity/v3/projects/22554735f59a426ea67d528448c2727d/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:14.950804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385523c-bb48-4613-9f58-c10147dcfecc None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.951343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385523c-bb48-4613-9f58-c10147dcfecc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.952526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9385523c-bb48-4613-9f58-c10147dcfecc None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.955071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 202/398] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:14.962396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.963347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.963853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.974440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:14.991609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:14.993337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:14.997490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:14.998176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:14.998445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:14.998760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:14.999437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:14.999786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.001215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.014914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 197/399] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.021708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.135516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 203/400] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:15.190740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c743833-4515-43dc-a448-03409c50d733 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.227275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.227922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.228203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.228475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.229611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.231080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.233120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 198/401] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.239041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.401632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88128f62-2f15-47e9-91f6-0c413da23294 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.407218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.417293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.471116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.471852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.472581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.472872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.474120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.476431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.478098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 204/402] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:15.484104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.528617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.599041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc45405f-eb3d-48a4-b31e-c705e33fe2be None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.625906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c1226-3c43-407c-85b6-d4d85bf913a4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.642167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73138425-6efa-4dde-8abc-ce9f5be7c03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.643062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73138425-6efa-4dde-8abc-ce9f5be7c03e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.643350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73138425-6efa-4dde-8abc-ce9f5be7c03e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.643623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73138425-6efa-4dde-8abc-ce9f5be7c03e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.656293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73138425-6efa-4dde-8abc-ce9f5be7c03e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-73138425-6efa-4dde-8abc-ce9f5be7c03e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:15.663201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:15.664074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:15.664855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 199/403] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:15.665521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b91e50-6139-49d3-985f-2599649f218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.665521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b91e50-6139-49d3-985f-2599649f218e None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.669022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a0d8c-10d1-4e76-bd61-0717c1e04da2 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.669022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a0d8c-10d1-4e76-bd61-0717c1e04da2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.669200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b91e50-6139-49d3-985f-2599649f218e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.669288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8a0d8c-10d1-4e76-bd61-0717c1e04da2 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.669358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b91e50-6139-49d3-985f-2599649f218e None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.669434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 200/404] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:15.671838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a0d8c-10d1-4e76-bd61-0717c1e04da2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.674695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b91e50-6139-49d3-985f-2599649f218e None admin] RBAC: Authorizing `identity:create_grant(project_id=69a6097d627b4c9da43f6fb3ca86f1ca, group_id=90bb2e95f23b4bdb8ecebfc098856afe, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.676549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b91e50-6139-49d3-985f-2599649f218e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.692526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 205/405] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 19:37:15 2022] PUT /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:15.694057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.694353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.694663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.716448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:15.717704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:15.749687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.785828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8a0d8c-10d1-4e76-bd61-0717c1e04da2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:15.800436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:15.803989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 206/406] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:15.824499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:15.825516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:15.826018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:15.826484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:15.827392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:15.827958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:15.829683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:15.841086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 201/407] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:15.846678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:15.975082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b250c7e-2b46-4d75-8d06-2deccf2aa79f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.011822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.012509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.012790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.013065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.014209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.015700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.017331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 207/408] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:15 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.023325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.106293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.159490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.179811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d657b4e0-762e-4840-8c96-0d671ea97e78 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.250483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f2d34-12ba-4750-8485-c6e1f6c0c918 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.255794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.256429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.256778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.257057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.262474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.263920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.265905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 202/409] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.271678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.288899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.289534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.289838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.290112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.302067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:16.306974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:16.307924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:16.309109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 208/410] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:16.318612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.346123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.391266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053fe764-eb07-4c6f-91d8-e98d98e19999 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.431473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.432124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.432392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.432655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.436194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] RBAC: Authorizing `identity:create_grant(project_id=69a6097d627b4c9da43f6fb3ca86f1ca, group_id=90bb2e95f23b4bdb8ecebfc098856afe, role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.437922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.472959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 203/411] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 19:37:16 2022] PUT /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca/groups/90bb2e95f23b4bdb8ecebfc098856afe/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:16.484540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.496718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f542b-abcd-4b33-be58-c9e00c643343 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.539481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.540485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.540967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.541440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.542594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:16.543214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.544861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.556231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 209/412] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.561597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.696716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba071-84af-448c-abae-c51c153f9939 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:16.698520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:16.740518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057356c0-7b54-45f2-9c68-c0c0d8feb357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:16.741224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057356c0-7b54-45f2-9c68-c0c0d8feb357 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.741497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057356c0-7b54-45f2-9c68-c0c0d8feb357 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.741785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057356c0-7b54-45f2-9c68-c0c0d8feb357 None admin] PATH_INFO: `/v3/groups/6ff62cc132a04937af9be06e4a61b954` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.743031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057356c0-7b54-45f2-9c68-c0c0d8feb357 None admin] RBAC: Authorizing `identity:get_group(group_id=6ff62cc132a04937af9be06e4a61b954)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:16.744476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057356c0-7b54-45f2-9c68-c0c0d8feb357 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:16.746451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 204/413] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/groups/6ff62cc132a04937af9be06e4a61b954 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:16.751657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:16.852368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c5a5f-c8df-452a-839d-b8d71cfba806 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.852662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c5a5f-c8df-452a-839d-b8d71cfba806 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.852921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9c5a5f-c8df-452a-839d-b8d71cfba806 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.854559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 205/414] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:16.859699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:16.859974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:16.860245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:16.890381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:16.891572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:16.933975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa1028-de42-48ab-b085-13a3211d502e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.001940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.002591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.002915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.003189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] PATH_INFO: `/v3/projects/35cfa2e060644668a62b0b52ed4b9871` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.005116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] RBAC: Authorizing `identity:get_project(project_id=35cfa2e060644668a62b0b52ed4b9871)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.006588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.010626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 210/415] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:16 2022] GET /identity/v3/projects/35cfa2e060644668a62b0b52ed4b9871 => generated 344 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.022386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.023590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 206/416] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:17.066077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.182364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f75ce80-d8b2-4c93-acc4-9fc829d9388c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.227847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.228513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.228789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.229065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] PATH_INFO: `/v3/projects/35cfa2e060644668a62b0b52ed4b9871/groups/6ff62cc132a04937af9be06e4a61b954/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.230964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] RBAC: Authorizing `identity:create_grant(project_id=35cfa2e060644668a62b0b52ed4b9871, group_id=6ff62cc132a04937af9be06e4a61b954, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.232391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.259533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 211/417] 10.222.0.15 () {58 vars in 1442 bytes} [Tue Sep 13 19:37:17 2022] PUT /identity/v3/projects/35cfa2e060644668a62b0b52ed4b9871/groups/6ff62cc132a04937af9be06e4a61b954/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:17.358264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.489092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.493244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8101a1e5-5ad2-4bb4-8cf0-5c2c710a4fb2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.531113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301cd7b-0cc0-47df-ad8f-e2afc6020603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.531746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301cd7b-0cc0-47df-ad8f-e2afc6020603 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.532022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301cd7b-0cc0-47df-ad8f-e2afc6020603 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.532300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301cd7b-0cc0-47df-ad8f-e2afc6020603 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.542895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b301cd7b-0cc0-47df-ad8f-e2afc6020603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b301cd7b-0cc0-47df-ad8f-e2afc6020603 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:17.547746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:17.548639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:17.549498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 207/418] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:17.555810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.674026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:17.699955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eb945e-b94b-4c41-a794-374581f21262 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.702985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d31c6-8488-402b-8abe-52aaa45116d4 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.703261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d31c6-8488-402b-8abe-52aaa45116d4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.703539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39d31c6-8488-402b-8abe-52aaa45116d4 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.705906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 208/419] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:17.712187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.712464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.712736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.739420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:17.740626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:17.761835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.762605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.762891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.763165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:17.764246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:17.765681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:17.774576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 212/420] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:17.780095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:17.897549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 209/421] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:17.916602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e66021-72c4-4da8-b8af-7ce66ace5b32 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:17.965897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:17.966880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:17.967375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:17.967844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:17.984704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.061168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.063302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.194256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 213/422] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:18.198804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.200141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.266268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.345764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.367487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c287c17-a9a4-4411-8b76-bf09e6b2b7cb None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.369317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78a70e5-f48f-4209-b056-65c26f2d07d2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.431447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.432080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.432354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.432624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.434006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.435623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.436243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.436510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.437024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.438539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.440397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 214/423] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:18.446090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.463728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:18.468890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:18.469921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:18.471621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 210/424] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:18.479016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.609127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de0a95c-6821-4ecc-be48-afc5f92de3ed None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.621565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.650650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de08391-9e16-45a2-99a9-8cf402c0f8e7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.666445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.667093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.667372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.667645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.668791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.670251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.672128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 215/425] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:18.677522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.685767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.686377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.686901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.687169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.687830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.688124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.689487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.698732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 211/426] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:18.704780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:18.724422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:18.828780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cce555-ce1a-4e9e-bdcb-8daad5f3ebcb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.868178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.869113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.869591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.870108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.874462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68faa6d7-d01b-4ca0-a1ab-f953f295a94f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:18.914389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:18.915005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.915285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.915558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.916287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=22554735f59a426ea67d528448c2727d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:18.916989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.918756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbaa6e4-57dd-498a-afe6-236fe75b96a4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.926703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 216/427] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 611 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:18.928458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a968342-e9e7-4da2-80e9-aa1927f99451 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.928719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a968342-e9e7-4da2-80e9-aa1927f99451 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.928996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a968342-e9e7-4da2-80e9-aa1927f99451 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.934829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 217/428] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:18.937515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:18.937823 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:18.938098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:18.955853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:18.957299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:18.959688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:18.961214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:18.975199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.007698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.079172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 212/429] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:19.083402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.105471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 218/430] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:19.202825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b9b4c-29cf-4c67-be09-0573cc45de76 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.218440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.240580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9caee37-02f6-496c-84a3-539c83354719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.241213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caee37-02f6-496c-84a3-539c83354719 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.241488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caee37-02f6-496c-84a3-539c83354719 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.241774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9caee37-02f6-496c-84a3-539c83354719 None admin] PATH_INFO: `/v3/projects/22554735f59a426ea67d528448c2727d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.242910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caee37-02f6-496c-84a3-539c83354719 None admin] RBAC: Authorizing `identity:get_project(project_id=22554735f59a426ea67d528448c2727d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.244320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9caee37-02f6-496c-84a3-539c83354719 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.246362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 213/431] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/projects/22554735f59a426ea67d528448c2727d => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.252773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.295993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4944f-507c-492a-bbc2-c9dee62d3168 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.296319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4944f-507c-492a-bbc2-c9dee62d3168 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.296548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4944f-507c-492a-bbc2-c9dee62d3168 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.298230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 214/432] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:19.308272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.308536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.308807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.329749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:19.330979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:19.368835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f23b5-aa9e-497f-ad72-e9c74e77e167 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.415130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 215/433] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:19.416155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.419612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.420255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.420527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.420791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.425760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.425760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.426999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 219/434] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.434707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.480074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.535210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c1083-cb54-492a-927a-3cd0bf06b972 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.548795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.558762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e743f6d-4e4d-46f0-a9d7-7d80ed15a2b8 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.576715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.577345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.577621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.577938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.590747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:19.596518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:19.597428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:19.598269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 216/435] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:19.603354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.604456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.605221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.605640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.606087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.607125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=22554735f59a426ea67d528448c2727d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:19.607579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.609992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.618487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 220/436] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=22554735f59a426ea67d528448c2727d&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 611 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.693283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.734841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ff146a-8350-4995-8428-d00d0c8205bf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.778682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.779795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.780312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.780787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.781671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] RBAC: Adding query filter params (name=member) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:19.782295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:19.784536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:19.799766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 217/437] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:19.801166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.817291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96560c17-ed53-40ed-9279-b5e3fb50f1e3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.819778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.839731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:19.867871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6391de-3919-43ba-b32a-0c764809370c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.868781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6391de-3919-43ba-b32a-0c764809370c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.869054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6391de-3919-43ba-b32a-0c764809370c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.869321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6391de-3919-43ba-b32a-0c764809370c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.884031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6391de-3919-43ba-b32a-0c764809370c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-de6391de-3919-43ba-b32a-0c764809370c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:19.889475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:19.890424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:19.891254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 221/438] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:19.891918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff2160-de4f-4bb1-8209-4c810b3f1260 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.892934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff2160-de4f-4bb1-8209-4c810b3f1260 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.892934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ff2160-de4f-4bb1-8209-4c810b3f1260 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:19.895665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 222/439] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:19.897181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff2160-de4f-4bb1-8209-4c810b3f1260 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:19.924728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b716f-73cd-4341-9398-c232d163506a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:19.964832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:19.965687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:19.965983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:19.966256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.015053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ff2160-de4f-4bb1-8209-4c810b3f1260 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.050806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.052251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.053616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.054820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.054820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.054820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.055423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.055726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.057138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b90c4c-8423-472f-983b-7896e81c75f5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.067906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 223/440] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:20.069210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.069501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.069847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.086639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.091291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:20.091291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:20.112841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.172531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 224/441] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:20.173611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.176046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 218/442] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:20.181812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.285831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f863a80f-5dae-45b5-a8b4-1b16339d30a9 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.300183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317adb8-ceba-40b3-af4e-4754bd6f98f9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.321782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.322685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.323159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.323645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.330730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.336972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.337594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.337907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.338181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.339298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.340725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.342380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 219/443] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:20.348210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.350874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.397611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.399438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.448895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0ec68-7cf9-42af-8157-dbc28d43708f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.483154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.483877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.484154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.484421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.485519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.487579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.489131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 220/444] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.490048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.500373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 225/445] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:20.501073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.554386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.559283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.591855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da3425-4901-459f-9406-5d2379b77bad None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.596738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6707fbd-bfe0-4970-b9eb-85e04c73055f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.625223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.625866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.626139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.626399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.630445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.631112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.631393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.631662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.632380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=61f630f0ba254ac5bb25aa078b9876a0, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.632688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.634132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.636261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:20.639786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 226/446] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=61f630f0ba254ac5bb25aa078b9876a0 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.640384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:20.640989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:20.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:20.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:20.645884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 221/447] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:20.651053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.758129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fafe1-8f23-492a-ad0d-2756c81d467e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.769318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd039f96-4dab-4738-8e63-659353fc011c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.779900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.796974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.797621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.797931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.798208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] PATH_INFO: `/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.799384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] RBAC: Authorizing `identity:get_project(project_id=69a6097d627b4c9da43f6fb3ca86f1ca)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.800838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.802420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:20.807519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 227/448] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/projects/69a6097d627b4c9da43f6fb3ca86f1ca => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.808210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.812952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.813555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.813863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.814233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.814902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:20.815206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.816572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.826335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 222/449] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:20.832323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.932286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f3b7cc-7e2c-4fa5-b3eb-daa80c15f564 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.953406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4912583-62c5-41b6-8cdf-6b9646f7e0f0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:20.969308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.970005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.970833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.971109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] PATH_INFO: `/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:20.972337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] RBAC: Authorizing `identity:get_group(group_id=90bb2e95f23b4bdb8ecebfc098856afe)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:20.973793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:20.975363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 228/450] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/groups/90bb2e95f23b4bdb8ecebfc098856afe => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:20.981428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:20.993312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:20.993945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:20.994234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:20.994484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.042394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.056639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.069555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.071016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.086690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa8075-905f-413f-9021-1ae261984ba3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.118455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.119050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.119326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.119606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.120323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] RBAC: Adding query filter params (group.id=90bb2e95f23b4bdb8ecebfc098856afe, role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:21.120626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.122079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.128286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 229/451] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/role_assignments?group.id=90bb2e95f23b4bdb8ecebfc098856afe&scope.project.id=69a6097d627b4c9da43f6fb3ca86f1ca&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:21.169147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 223/452] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:21.174786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.278414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a8d7ee-fc69-4578-9b6b-758a58e48e39 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.298469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.315458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.316186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.316465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.316740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] PATH_INFO: `/v3/projects/35cfa2e060644668a62b0b52ed4b9871` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.317883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] RBAC: Authorizing `identity:get_project(project_id=35cfa2e060644668a62b0b52ed4b9871)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.319320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.320996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 230/453] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/projects/35cfa2e060644668a62b0b52ed4b9871 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:21.326527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.416504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:21.427334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730c254-77cf-48ff-9c7b-8ed29e4eb1a0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.462778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c30ae-5995-443d-8667-d09409055dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.463403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c30ae-5995-443d-8667-d09409055dcb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.463680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c30ae-5995-443d-8667-d09409055dcb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.463948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c30ae-5995-443d-8667-d09409055dcb None admin] PATH_INFO: `/v3/groups/6ff62cc132a04937af9be06e4a61b954` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.465066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c30ae-5995-443d-8667-d09409055dcb None admin] RBAC: Authorizing `identity:get_group(group_id=6ff62cc132a04937af9be06e4a61b954)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.466525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c30ae-5995-443d-8667-d09409055dcb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.468218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 224/454] 10.222.0.15 () {56 vars in 1180 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/groups/6ff62cc132a04937af9be06e4a61b954 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:21.474384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:21.572522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44397af3-bbb5-4260-929f-d1804fa44e77 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:21.603663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:21.604251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:21.604527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:21.604798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:21.605503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] RBAC: Adding query filter params (group.id=6ff62cc132a04937af9be06e4a61b954, role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=35cfa2e060644668a62b0b52ed4b9871) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:21.605839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:21.607243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85136e09-4a4e-43b8-a66c-c6d28b131e06 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:21.613296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 231/455] 10.222.0.15 () {56 vars in 1376 bytes} [Tue Sep 13 19:37:21 2022] GET /identity/v3/role_assignments?group.id=6ff62cc132a04937af9be06e4a61b954&scope.project.id=35cfa2e060644668a62b0b52ed4b9871&role.id=40a9647e8ec745c999c2457d053557a7 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:24.251650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38b12-022b-4ace-a421-925920cfda10 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.252491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38b12-022b-4ace-a421-925920cfda10 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.252930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c38b12-022b-4ace-a421-925920cfda10 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.254671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 225/456] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:24.263425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631163e-d4d8-4a34-a06f-b767ec9a1963 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.263425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631163e-d4d8-4a34-a06f-b767ec9a1963 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.263425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4631163e-d4d8-4a34-a06f-b767ec9a1963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.286736 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4631163e-d4d8-4a34-a06f-b767ec9a1963 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:24.288334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4631163e-d4d8-4a34-a06f-b767ec9a1963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:24.293101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cf95b-0cbf-4980-9452-fc2a1b07f4c4 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.293630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cf95b-0cbf-4980-9452-fc2a1b07f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.294130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14cf95b-0cbf-4980-9452-fc2a1b07f4c4 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.295881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 226/457] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:24.301561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02103c-268b-4ff1-9a6c-bbfa04c9af50 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.302110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02103c-268b-4ff1-9a6c-bbfa04c9af50 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.302583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02103c-268b-4ff1-9a6c-bbfa04c9af50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.333768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9d02103c-268b-4ff1-9a6c-bbfa04c9af50 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:24.336067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9d02103c-268b-4ff1-9a6c-bbfa04c9af50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:24.362144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:24.373088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 232/458] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:24.460005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 227/459] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:24.495846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714686a2-1deb-49b9-9b02-0e9de4f99ddb None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.496179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714686a2-1deb-49b9-9b02-0e9de4f99ddb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.496579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714686a2-1deb-49b9-9b02-0e9de4f99ddb None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.498108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 233/460] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:24.507609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.507822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.508084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.526435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:24.527602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:24.624100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 228/461] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:24.691501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714686a2-1deb-49b9-9b02-0e9de4f99ddb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:24.757263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:24.776829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:24.878598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e7147-627f-4064-bc0d-42eb5acca547 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:24.890987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714686a2-1deb-49b9-9b02-0e9de4f99ddb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:24.916042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:24.917147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.917634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.918129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.930093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:24.932271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:24.950807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:24.951798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:24.952652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:24.952652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:24.952652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:24.955172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 229/462] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:24.961319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:24.973976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:24.974608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:24.974879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:24.975146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:24.986507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:24.987941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:25.003466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:25.004380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:25.005239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:25.005239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:25.005239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:25.009622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 234/463] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:25.010641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.051699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.055711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.088730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef448e1-eff3-44a9-a160-8ce3006edc0b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.125133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8188ae-6c61-4e87-bc60-c5d19afa0b1f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.125459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.126070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.126350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.126616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.127410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:25.127856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.129353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.142058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 230/464] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:25.147157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.162589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.163220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.163496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.163761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.174815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.176237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-028bd1bf-8237-4e89-b0d9-412ee41ea9ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:25.190430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:25.191454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:25.192323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:25.192323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:25.192323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:25.192323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 235/465] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:25.192998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28750-498b-4726-b229-2352197f9beb None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.193103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28750-498b-4726-b229-2352197f9beb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.193572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c28750-498b-4726-b229-2352197f9beb None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.194664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 236/466] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:25.199361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28750-498b-4726-b229-2352197f9beb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.254318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b00fe00-cccc-48e5-95ea-a2c814a6e8f1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.285478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.292537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.293296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.293573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.294072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.294490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.294796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:25.295098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.296451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.313721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 231/467] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:25.313721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.315217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c28750-498b-4726-b229-2352197f9beb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.352497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2982bfb6-53af-4d88-aa4d-67b70088936e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.353097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982bfb6-53af-4d88-aa4d-67b70088936e None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.353370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982bfb6-53af-4d88-aa4d-67b70088936e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.353646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2982bfb6-53af-4d88-aa4d-67b70088936e None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.354396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982bfb6-53af-4d88-aa4d-67b70088936e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.355818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2982bfb6-53af-4d88-aa4d-67b70088936e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.389764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 237/468] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:25 2022] POST /identity/v3/users => generated 246 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:25.389764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.389764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.389764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.399789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:25.401266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:25.436933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08153d45-fa19-4233-90d4-f035a9095ecd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.482174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.483360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.483843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.484380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.485614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:25.486264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.487697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 238/469] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:25.488261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.488541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.503775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 232/470] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:25.509942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.592918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.618219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:25.637145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d48ee-aea8-466d-b44a-e68f517f7d82 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.682263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460f4dd-0640-4e0f-b1d3-48ad7f25f6de None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.684454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.685545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.686071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.686534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.687397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.689535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.719060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 239/471] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:25 2022] POST /identity/v3/users => generated 248 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:25.719702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:25.720426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:25.720662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:25.720982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:25.721641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:25.723121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:25.748893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 233/472] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:25 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:25.841539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:25.964450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93c808-d7e2-4647-b991-031b11853462 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:25.996419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.007322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.007993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.008261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.008535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.023007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.024481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-53a177ec-6875-4c40-9908-e825cf41bfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:26.040468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:26.041401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:26.042264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:26.042264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:26.042264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:26.042264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 240/473] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:26.048063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.129377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c955b-c752-4951-a5a5-49aaf82bcba3 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.129688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c955b-c752-4951-a5a5-49aaf82bcba3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.129953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c955b-c752-4951-a5a5-49aaf82bcba3 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.131620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 241/474] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:26.136996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.137403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.137677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.168154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a584830-a11b-4554-a7bb-9fac1e3fccbd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.179045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:26.180291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:26.182524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.231623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.232148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.232424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.232693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.233338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:26.233637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.235569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.253912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 234/475] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:26.259452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.324926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 242/476] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:26.394800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3c8183-c762-47e4-868a-07f2cccea088 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.442474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.443082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.443357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.443627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.444281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.445696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.474525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 235/477] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/users => generated 249 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:37:26.748828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.869695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf3c6dd-90e4-413a-95f0-131c6867ece0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:26.880708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:26.915291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:26.915903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:26.916181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:26.916450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:26.917136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:26.918527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:26.931965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.sql.core [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:37:26.931965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:37:26.931965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [parameters: {'id': '921f272b69ec40c3b5092c2edf1aca49', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:37:26.931965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93112) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application result.read() Sep 13 19:37:26.948678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:37:26.949594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application yield resource Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise self.value Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application session.commit() Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:37:26.950635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:37:26.951808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:37:26.952692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application result.read() Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:37:26.954128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [parameters: {'id': '921f272b69ec40c3b5092c2edf1aca49', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:26.954911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 19:37:26.958136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:26.959887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 243/478] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 19:37:26 2022] POST /identity/v3/roles => generated 144 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:26.959993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:26.991807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:27.076041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416bbcd-97f7-4ce0-b679-3e10aee6e4ef None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.115409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92455e7b-d05c-46b3-ac54-5b516439299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.116057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92455e7b-d05c-46b3-ac54-5b516439299e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.116334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92455e7b-d05c-46b3-ac54-5b516439299e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.116605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92455e7b-d05c-46b3-ac54-5b516439299e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.128198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92455e7b-d05c-46b3-ac54-5b516439299e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-92455e7b-d05c-46b3-ac54-5b516439299e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:27.138466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:27.141299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:27.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 236/479] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:37:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:27.148973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:27.299085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bcdd50-e667-45fb-8787-2a6c768b8203 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:27.355396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:27.356809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:27.357292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:27.357772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:27.358628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:27.359159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:27.360727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f56b6-f832-4289-9e4b-84b822214ae4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:27.372875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 244/480] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:28.289766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a907c5c-4be4-42a8-9908-d434946b26a3 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.290295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a907c5c-4be4-42a8-9908-d434946b26a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.290295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a907c5c-4be4-42a8-9908-d434946b26a3 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.291891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 237/481] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:28.297043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f52cd-8b28-4555-a548-83e2306f5fbd None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.297336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f52cd-8b28-4555-a548-83e2306f5fbd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.297617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212f52cd-8b28-4555-a548-83e2306f5fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.323706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-212f52cd-8b28-4555-a548-83e2306f5fbd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:28.325258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-212f52cd-8b28-4555-a548-83e2306f5fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:28.352641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e821412-460b-416d-aa4d-196d2b56dbf6 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.352915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e821412-460b-416d-aa4d-196d2b56dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.353352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e821412-460b-416d-aa4d-196d2b56dbf6 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.355183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 238/482] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:28.367196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.367905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.368546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.390610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.394973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:28.396175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:28.459363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 245/483] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:28.467133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216af863-684c-45b0-bf3d-7765060ad3f9 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.467530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216af863-684c-45b0-bf3d-7765060ad3f9 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.467880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216af863-684c-45b0-bf3d-7765060ad3f9 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.470130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 246/484] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:28.475100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.475456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.475804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.501264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 239/485] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:28.511575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:28.512768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:28.628242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 247/486] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:28.762993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.779626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:28.883897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:28.897717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a15d7-b2e0-4dc7-8893-67659e7283a3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:28.961122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:28.961795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:28.962070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:28.962339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:28.980709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:28.986469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:28.987416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:28.988211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:28.991864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 240/487] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:28.992930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.021613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.030668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15833b8-a394-4e8f-a1e8-adea44354b62 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.073026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.073656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.074260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.074553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.083649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.087452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:29.093154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.094085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:29.094902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:29.097805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 248/488] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.098822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.128496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b72e32-1b68-418d-8010-d28ff1e1dd7b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.169224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.170384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.170925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.171893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.184368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:29.191145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.192018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:29.192841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:29.195291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 241/489] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.196268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.224512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388b8d5-19ee-4baa-9831-7ebbfe3ced62 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.268325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.269362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.269866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.270338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.271190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.271745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.273323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.284279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 249/490] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.285360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.310453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.322291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819cf86-9ab0-4216-84f2-691824e37edf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.364786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.375344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.376196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.376612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.377351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.378527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.379132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.380994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.397103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 242/491] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.397103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.407857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a40923-149b-4f59-b0fb-9594b5527f55 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.455933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.457128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.457609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.458522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.459584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.459894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.461260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.470128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 250/492] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.471309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.512375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8972de-f4dd-42d4-8fb4-b5da864c6337 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.549534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.550745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.551473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.551940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.552986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.564657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.566188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.573995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a585f8-2aa2-456f-baf5-bc970dad0558 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-05b1a9f8-49b7-4a4c-a92c-ee73ba5840df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:29.579957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.580853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:29.581706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:29.581706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.581706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:29.581706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 243/493] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.582694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b543e9-5245-49ec-a0e0-ce4d1b012794 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.582694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b543e9-5245-49ec-a0e0-ce4d1b012794 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.582694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b543e9-5245-49ec-a0e0-ce4d1b012794 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.584037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 244/494] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:29.584535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b543e9-5245-49ec-a0e0-ce4d1b012794 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.617529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.618214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.618494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.618764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.629720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.631172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.635726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:29.645515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.646490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:29.647415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:29.647415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.647415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:29.647415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 251/495] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.648215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.704927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b543e9-5245-49ec-a0e0-ce4d1b012794 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.741019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.741835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.742115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.742551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.745291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523beb7-2019-452a-9dfa-9a277ad85986 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.753867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.755438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-15c972e5-0e46-40b4-8177-81122596cec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:29.771538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:29.772453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:29.773294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:29.773294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:29.773294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:29.773294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 245/496] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:29.773923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.774187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.774457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.783132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.789540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.790188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.790467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.790739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.791381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.791683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.793139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.797189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:29.798667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:29.805388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 252/497] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.806222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.882656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 246/498] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:29.884101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:29.918424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:29.922192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179fde01-8051-4e85-908a-5cf91262e429 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:29.963042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:29.963788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:29.964203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:29.964481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:29.965205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:29.965509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:29.968614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:29.988080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 253/499] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:29.988080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.007116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f201c-da16-4aca-9e70-255b7d0b485a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.047053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.047874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.048300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.048716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.049632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.050390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.051907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.053008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.067708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 247/500] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.068814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.119211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c83fba-1daa-4ef4-a27e-0b25d33a8c5c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.167709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.168376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.168657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.168927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.177231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.178714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.179307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:30.192541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:30.193504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:30.199273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 254/501] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:30.199998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7afbc-0298-4ed3-bb62-d0a22e1a0e6e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.200084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.240944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.242104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.242532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.243050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.253112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.254862 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:30.270597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:30.271628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 248/502] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:30.272511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.300879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff621b8-718d-4b73-8282-e44db8098230 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.314205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.335656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.336286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.336565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.336837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.344094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.345526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:30.356412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:30.357508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 255/503] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:30.358520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.377317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf671ab-7b64-4474-854e-0c468b3d388c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.410422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3243dbf8-e899-4c49-b2c9-ea6c80ca9069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.411034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243dbf8-e899-4c49-b2c9-ea6c80ca9069 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.411309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243dbf8-e899-4c49-b2c9-ea6c80ca9069 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.411578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3243dbf8-e899-4c49-b2c9-ea6c80ca9069 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.421969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3243dbf8-e899-4c49-b2c9-ea6c80ca9069 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3243dbf8-e899-4c49-b2c9-ea6c80ca9069 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:30.427095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:30.428092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:30.428889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 249/504] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:30.429561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d35ba6-e0b6-47c7-b486-4eb9c7a760fd None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.429625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d35ba6-e0b6-47c7-b486-4eb9c7a760fd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.430004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d35ba6-e0b6-47c7-b486-4eb9c7a760fd None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.431238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 250/505] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:30.431833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d35ba6-e0b6-47c7-b486-4eb9c7a760fd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.443954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.459093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09895a-d935-4987-82f0-36110f7586f4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.491911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.492508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.492781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.493052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.493795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.494101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.495515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.531117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 256/506] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.531883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.532598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d35ba6-e0b6-47c7-b486-4eb9c7a760fd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.561053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.568134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.568730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.569006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.569273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.570029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.570329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.571739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10beb068-1fab-4c1d-a018-09d3a31152a2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.606963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 251/507] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.608757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.609044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.609319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.626904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:30.628082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:30.630075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99df5512-17d0-4bc5-a0af-eff861952fd1 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.664594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.665191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.665468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.665752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.666471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.666773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.668196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.685521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 257/508] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.686942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.687344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.696069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 252/509] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:30.697344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.783372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.797054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83902534-65d8-4ddf-aa99-cd1770b9c4f8 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.803308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1906b86d-14e8-4cee-9e45-e723336be622 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.830726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.831334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.831612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.831882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.832537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:30.832837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.834232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.848886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 258/510] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:30.848886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:30.854612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.855219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.855486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.855752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:30.920224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:30.928555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:30.930183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:30.947937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395a5465-2c17-4e85-8997-d53430ca35a7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:30.980855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:30.981452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:30.981726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:30.982016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.000722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.032450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 253/511] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:31.033279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.051976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.053370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.124051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.130730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b27b16-7eb3-423b-a347-5555a036e178 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.149496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 259/512] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:31.150343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.165281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.166280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.166560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.166829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.198093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.233511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.234955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.247128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b251c3f3-67e5-496d-93aa-1b9c453fe13a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.279403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.280570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.280842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.281105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.290622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.292046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:31.308136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.309320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:31.310316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:31.310316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.310316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:31.310316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 260/513] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:31.310579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.330375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.339901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 254/514] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:31.340690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.414075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610471e-488a-4f50-849c-32e9b3e89c38 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.415552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.436679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27104a89-3b72-474f-b289-03d4ec0c917e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.447829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.448453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.448731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.449006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.460645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.462134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.470884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.471507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.471788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.472059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:31.476525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.477570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:31.478475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:31.478475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.478475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:31.478475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 261/515] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:31.478475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.483635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.485083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:31.500149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.501061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:31.501949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:31.501949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.501949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:31.501949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 255/516] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:31.502177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.548464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.580527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1c670a-10bf-4929-a914-83a5f775830a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.599896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2d8621-2cb8-432f-8cee-b27148acb62f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.612280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.612897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.613171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.613438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.624254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.625696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.632469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.633052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.633327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.633603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.634263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:31.634560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.635896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:31.640137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.641050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:31.641946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:31.641946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.641946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:31.641946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 262/517] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:31.642159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.648461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 256/518] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:31.649245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.652213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.743653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091d120-c6a8-4b0e-9226-7fd9baacb93d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.743882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273fbe1-e67f-4485-bb6c-a21d8c2e3d15 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.754340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.777190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.777841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.778170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.778384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.779984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.780575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.780886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.781147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.781813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:31.782114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.783483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.788590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.790037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.794337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 257/519] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:31.795110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:31.806991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:31.807967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:31.809066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:31.809066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:31.809066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:31.813339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 263/520] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:31.814049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.872396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.895243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff9728b-bc4e-4640-b77a-8340d2b80cc2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.924779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3544c552-4407-46ec-8ef1-5a7e0eb8d9d4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:31.928293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.928889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.929164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.929511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.930190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:31.930489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.931841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.944227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 258/521] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:31.945000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:31.958114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:31.958886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:31.959177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:31.959458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:31.960471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:31.960928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:31.962332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:31.971812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:31.973942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 264/522] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:31.974791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.041361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e696ec6-26bc-4d1b-86d0-445311abf3e6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.074589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.075050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe6fbd-6628-458e-8659-7af119bc733e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.075199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.075473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.075738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.082829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.091605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:32.095521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:32.096414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 259/523] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:32.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.112238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.112850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.113132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.113405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.114096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.115520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.137214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 265/524] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:31 2022] POST /identity/v3/users => generated 247 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:32.137956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.202617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.216612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04244a93-ea97-44c3-a6fd-bbe95288d34d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.253521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.254162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.254442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.254711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.255362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.255670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.257058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.270104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 260/525] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.270969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.271606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce628fb2-893d-4dea-a6df-34ef42c4458a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.311778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.312474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.313429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.313429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.320670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.322127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:32.335666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:32.336528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:32.337350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 266/526] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:32.337853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.377228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.391588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b68855-e037-40e2-a70f-c1960b8a74b3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.431308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.432001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.432209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.432452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.439780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.441468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.450339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7acdb-55b1-4630-81ee-d3b26a7d9d4f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:32.454704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:32.455563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 261/527] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:32.461761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.470916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.494254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.495255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.495645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.495917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.496916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.497219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.498934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.529012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 267/528] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.531045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.589879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6dfcf5-8241-415f-83d1-87d6eebafabb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.628223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.628772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.629269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.629597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.636538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.636879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.638842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.650649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f26e3c2-4e69-4239-8f84-d928ae525256 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:32.651347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:32.652163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:32.653380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 262/529] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:32.657557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.687407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.688016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.688446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.688716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.689756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.690202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.691633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.714993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 268/530] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.714993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.737130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.774025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e025a39-90bc-41cb-b910-e079d6184e2c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.811178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.811774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.812046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.812309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.813043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.813353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.814811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.822302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f53f1-a749-4f0b-9d5a-43cdf3af9fa2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.845824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 263/531] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:32.845824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:32.867645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.868251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.868523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.868789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.892795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.945130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.946616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:32.952986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d55a70-21d9-469f-b9c4-5b6a3ce9d575 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:32.980916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:32.994557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:32.995152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:32.995423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:32.995685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:32.996368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:32.996760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:32.998729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.016518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 264/532] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:33.016518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.059004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 269/533] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:33.059878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.116502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.125600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8afb6d9-3e1d-4d5c-890b-522a66b747f6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.161528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358815a-15ed-4f90-b905-597844472555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.162081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358815a-15ed-4f90-b905-597844472555 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.162355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358815a-15ed-4f90-b905-597844472555 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.162624 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358815a-15ed-4f90-b905-597844472555 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.163383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358815a-15ed-4f90-b905-597844472555 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=910ca69bc9bf4f328d9537f0b9bb1658) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:33.163634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358815a-15ed-4f90-b905-597844472555 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.165040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358815a-15ed-4f90-b905-597844472555 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.167728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a567ff-633f-4a65-9742-f8cb228605d6 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.170997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 265/534] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/role_assignments?user.id=910ca69bc9bf4f328d9537f0b9bb1658&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:33.171956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358815a-15ed-4f90-b905-597844472555 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.223333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.223955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.224534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.224815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.225564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=0f8ea3cf81f944a2b02e59b3ff049f6d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:33.226935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.228480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.236408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 270/535] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/role_assignments?user.id=0f8ea3cf81f944a2b02e59b3ff049f6d&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:33.238670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.264258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.370433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358815a-15ed-4f90-b905-597844472555 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.434748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.435280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.435556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.435824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.436561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=c21dea41dba24bf88466a7c3f00b8c2b) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:33.436867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.438334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.450854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 266/536] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/role_assignments?user.id=c21dea41dba24bf88466a7c3f00b8c2b&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:33.453925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13886b9d-d97c-425a-8a02-412d0a6f5737 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.501623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.503526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.504556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.505146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.505675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.520689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.522578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:33.541084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:33.542314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:33.543476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:33.543476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:33.543476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:33.545430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 271/537] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:33.553340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.673944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c8a731-8564-4621-8418-d6151d06b759 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.691998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:33.712006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.712663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.712948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.713220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.713895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:33.714198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.715553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.729713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 267/538] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:33.736222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:33.886603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6a871-3d0c-4ed8-946e-361a704cac87 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:33.931144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:33.931850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:33.932135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:33.932408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:33.942168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:33.943559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:33.957106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:33.958014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:33.959437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 272/539] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:33.964061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.121390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3997009-d796-46e3-939b-ff8617f4fd41 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.192069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.193124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.193600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.194102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.195013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:34.195543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.197242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8e319e-b74e-4194-b819-0daf9938080b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.220886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 268/540] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:34.226697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:34.263965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:34.353595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7d97c6-5aee-47fa-8d10-6f9c125ad7d5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:34.398466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:34.399081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.399360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.399635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.400364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=36fab0873915497cada62e8ea73bd4d6) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:34.400668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:34.402135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065d61ce-7c5f-425c-97ec-038a5b4430f7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:34.412774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 273/541] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:34 2022] GET /identity/v3/role_assignments?user.id=36fab0873915497cada62e8ea73bd4d6&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:34.688926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fad6a-7f53-46cc-8df2-760712936093 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.689439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fad6a-7f53-46cc-8df2-760712936093 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.689439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4fad6a-7f53-46cc-8df2-760712936093 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.691096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 269/542] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:34.699607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:34.700029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:34.700221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:34.722694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:34.723916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:34.813888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 274/543] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:35.191597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fad6a-7f53-46cc-8df2-760712936093 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.237023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.318410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4fad6a-7f53-46cc-8df2-760712936093 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.374992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.375994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.376467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.376931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.392773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:35.397823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:35.398691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:35.399490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:35.402064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 270/544] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:35.407675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.448761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.542721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264a65d-2a5f-44fd-8fd8-abc0dbb0e31c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.587341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.588000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.588282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.588551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.589218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:35.589520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.590981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ed9f6-9bbb-4bb5-90fd-dc81cafb9b40 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.600917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 275/545] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:35.606640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.733446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:35.740030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe0a5b5-85b4-4bc2-8cf5-eae5b5852f27 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:35.783222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5d4cf-200f-401e-bb29-1b18082573e4 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.784263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5d4cf-200f-401e-bb29-1b18082573e4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.784862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5d4cf-200f-401e-bb29-1b18082573e4 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.786910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 276/546] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:35.789879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:35.789879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.789879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.789879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.792810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:35.793324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:35.793840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:35.801260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:35.802721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:35.815169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:35.816659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:35.817691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:35.818612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:35.819477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:35.819477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:35.819477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:35.819477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 271/547] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:35.827619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:35.952123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 277/548] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:36.005986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eccf0b3-2164-4c52-9055-fb639453d02e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.058951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.059587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.059866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.060186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.060883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.061187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.062660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a80a62c-f982-431b-a3d1-cc66de83b2d7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.082202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 272/549] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.087964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.119099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.215192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33d987-8413-4464-864c-65cc2811d8a3 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.215869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33d987-8413-4464-864c-65cc2811d8a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.216881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33d987-8413-4464-864c-65cc2811d8a3 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.225812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 273/550] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:36.225941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00705d5d-b8d2-4ac6-9226-c464cf800196 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.231399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.231683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.232526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.255346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:36.256727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:36.264239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.265682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.266346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.266674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.266999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] PATH_INFO: `/v3/users/swift` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.275056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.276629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5078996d-2924-49de-a05c-bace6ab73b1b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:36.289492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:36.290620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:36.291732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 278/551] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:36.292619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004fa06-7035-4173-b74a-79deaf003385 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.292619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004fa06-7035-4173-b74a-79deaf003385 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.292619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3004fa06-7035-4173-b74a-79deaf003385 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.295670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 279/552] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:36.299778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.300056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.300428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.319414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:36.320596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:36.332200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 274/553] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:36.333083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.401922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 280/554] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:36.402979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.406783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.433218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f824977-098e-4c1e-b97d-7aa43f1467a8 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.469205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.470038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.470314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.470582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.471304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:36.471603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.473055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.494082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.505769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 275/555] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:36.511384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.618604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8ab1e-20a7-4b5c-8c56-69b95c9291cb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.628037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b0f66-7ffd-4b4f-89a3-a2fa4f213dac None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.658339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.659377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.659775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.660073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.666769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.667484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.667828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.668250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.679511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:36.685036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:36.686886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:36.688902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:36.694503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 281/556] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:36.694503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.734096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.742342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:36.744054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:36.759868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.798476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0d4042-0d6e-4afe-8e45-6d4be6089b6f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.838077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:36.839413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:36.839413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:36.839588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:36.850304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:36.855155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:36.856056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:36.856920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 282/557] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:36.857591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.862968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 276/558] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:36.863915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:36.964795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537a040-1faf-4e04-ab01-4ab97bab3374 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.970142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:36.977754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cdbfd-f498-4549-984a-f0907151f06a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:36.993966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.005542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.006214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.006489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.006758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.007415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.007712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.009073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.014629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.014876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.015178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.015446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.021766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 283/559] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.021766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.025761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:37.031147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.032081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:37.032914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 277/560] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.033611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.121786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a8a31-97fb-48b8-848d-0fa1f16cb791 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.129310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8442c6a-7a7e-4b6a-9e5c-c0a6b03b6b88 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.157753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.158749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.159058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.159334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.160003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.160307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.161672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268fda1e-03f1-4475-b431-7fa4972851b1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.162771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.163383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.163656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.163930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.169831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 284/561] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.170977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770030-e748-4b79-98b1-2b667b386481 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.171263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770030-e748-4b79-98b1-2b667b386481 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.171539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770030-e748-4b79-98b1-2b667b386481 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.172732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 285/562] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:37.173416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81770030-e748-4b79-98b1-2b667b386481 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.178239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.179667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:37.194504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.195532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:37.196403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:37.196403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.196403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:37.197867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 278/563] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.198857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.210531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.225227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.276074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81770030-e748-4b79-98b1-2b667b386481 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.297279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0377d5b3-cd01-410f-a51c-706bb2e0ea90 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.309033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.309651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.309961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.310229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.320408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.321878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.330560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.331148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.331424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.331689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.332337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.332630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.333993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d86cb-fa47-486c-a853-70553a46b0ec None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:37.335964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.336892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:37.337793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:37.337793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.337793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:37.337793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 286/564] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.337793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.345521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 279/565] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.347213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.347512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.347785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.366543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:37.367716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:37.428837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.433542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.438552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 280/566] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:37.439485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.440438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e082d35-ec35-4176-a737-b08e47f98462 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.475715 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.476348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.476628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.476902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.487412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.488837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:37.503738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.504880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:37.505978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:37.505978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.505978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:37.511737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 287/567] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.511737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.546799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12ed73e-2c22-49aa-8386-c995923c5d04 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.585036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.585642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.585960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.586228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.586859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.587157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.588497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.601447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 281/568] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.606450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.614475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee08cc-090b-4d57-ba2d-5ccd392979eb None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.652031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.652790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.653071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.653346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.654053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.654357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.655718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.667402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 288/569] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.668203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.675904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.683539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.720786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e36176c-9f2a-41d5-bf3d-dbbed2e0dad2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.753640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.754288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.754561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.754834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.765720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.767164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.771666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457b7250-2a32-43fc-8400-bab3312f4606 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:37.781055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.782188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:37.783595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:37.783595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:37.783595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:37.783595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 282/570] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.783595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.808723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.809329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.809602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.810409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.811045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:37.811342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.812709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.823426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 289/571] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:37.824450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.882646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a6a6d-1d50-441b-9b3a-3463e6357972 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.895635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.898482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:37.917201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.917848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.918123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.918391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.925643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.927110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.927833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc6527b-7891-49f8-86a2-f9fc6608de1a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:37.938962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.940023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 283/572] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.941128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:37.964660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:37.965286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:37.965564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:37.966007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:37.973827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:37.975263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:37.986575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:37.987671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 290/573] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:37.988608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.041303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeeba90-5503-4747-b66a-d7796180200c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.076120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.076739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.077014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.077282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.086368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbd644-6d22-46db-93cc-18e113ad9929 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.087027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:38.092189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.093111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:38.093980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 284/574] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.094687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.117590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.123550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.124157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.124437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.124709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.125382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.125669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.127044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.127743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.138917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 291/575] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.138917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.196196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7285620-994d-48b2-829a-e07cbfb81dd2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.229527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.230165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.230438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.230705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.236033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926b5925-2195-4553-85cf-3d10358635ac None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.238579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.240019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:38.251535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.252483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:38.253449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 285/576] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.254243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.276402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.277327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.277616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.277928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.279146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.279452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.280877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.292047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 292/577] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.292832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.342874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.344484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.352049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d178e-a8d3-4045-a1dd-a5b891a9bfc9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.385081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.385691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.386000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.386273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.387000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.387310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.388929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.392741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d033fa38-3b04-44f5-a2cd-af5cd4a7f276 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.406057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 286/578] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.407152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.431166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.431860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.432139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.432675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.433329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.433629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.435055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.443287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 293/579] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.444056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.506189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea371a5-ced3-45ff-9c33-b54151f09ae1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.539045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.539664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.539939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.540208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] PATH_INFO: `/v3/users/glance` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.543057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a53889-6d69-47c8-a12b-2419ab747264 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.547550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.548972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.556542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:38.560726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.561902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 287/580] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.563140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.575060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.580345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.580978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.581277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.581532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.582278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.582583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.583991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.601358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 294/581] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.602170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.665463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecc548-18dc-47ea-bdcf-b9ab6c8e9b7f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.698107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.698834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.699189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.699342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54071f3-dbac-4af8-8b06-fcfd4988facc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.699579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.700852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=38d08c9aa2604871bec462424708b85b) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.701221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.702731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.707983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 288/582] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/role_assignments?user.id=38d08c9aa2604871bec462424708b85b&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.708743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.752167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.753342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.753621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.753942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.766227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.767636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:38.783967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.785002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:38.786352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:38.786352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:38.786352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:38.786352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 295/583] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.786352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.802568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.816653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:38.836294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3886d2a0-98fc-461e-806b-f1fbe955f53a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.875731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.876373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.876650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.876922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.889390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.897797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.903988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892a7f21-bc27-4c10-a1ae-47eae0432d97 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:38.906585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:38.907537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:38.908459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:38.908459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:38.908459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:38.908459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 289/584] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:38.908459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:38.941830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:38.942762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:38.943229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:38.943497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:38.944209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:38.944508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:38.945940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:38.963774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 296/585] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:38.964594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.019655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47883b9-48e1-4a88-b119-589af47a3707 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.048475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.050011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.059231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.060493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.060772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.061044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.072171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.073605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.074672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e5b0a1-a448-4a91-83d9-58e6655c2ef2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:39.088613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.089762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:39.090767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:39.090767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:39.090767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:39.090767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 290/586] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.090767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.118511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.119039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.119315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.119584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.120216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.120512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.122394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.133967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 297/587] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.134803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.208454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636d8f-0a36-4b29-937b-7c51040ead0c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.237410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb17317-a7c8-4140-a718-b173b4c45f3a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.242643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.243243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.243517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.243784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.244421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.244714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.246079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.258555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 291/588] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.259167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.277978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.278895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.279178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.279453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.289722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.291357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.293057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.300774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:39.308963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.310622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:39.311644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:39.311644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:39.311644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:39.311644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 298/589] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.311644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.375286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffedff62-57e2-49ff-aba0-c8960e806e08 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.415069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.415668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.415941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.416207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.416547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9ea7dc-4d9d-4fd5-9f25-e9d3a55bbca4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.416838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.417131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.418531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.435909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 292/590] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.435909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.458759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.459482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.459833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.460173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.467216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.468706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:39.480713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.482439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:39.483485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 299/591] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.484174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.542177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.545545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.549776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c49cad-9502-4c7e-bcba-85c57f4d4f3c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.601233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.602296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.602482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.602700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.603087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d08de97-1bc8-47c3-810b-8780be3a4b4e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.613402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.615522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:39.627972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.629165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:39.630218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 293/592] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.630853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.645108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.645719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.646025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.646614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.647591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.647893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.649438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.662507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 300/593] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.663467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.744043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca92ec5-69ca-4286-818e-56872e863463 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.765538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1934d-a324-45fe-bcee-455ca628a462 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.779259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.779870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.780152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.780424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.780588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.788298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.789717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:39.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:39.802506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:39.803858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:39.804354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.804354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.804354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.804354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.804639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.804639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.805451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.807651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 294/594] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:39.808349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.819346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 301/595] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.821193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.832285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:39.921553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410fcc-d5c1-40b9-b7b9-7e757a5ecfde None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.942160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917a2e7-16dc-4249-a5c4-f5f612082c6f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:39.956224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.956824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.957098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.957366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.958127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:39.958428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.959824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:39.976076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:39.976703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:39.976979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:39.977247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:39.982842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 295/596] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:39.982842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:39.990148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:39.991574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:40.002626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:40.003536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:40.004505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 302/597] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:40.005000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.045015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.064561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.088839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30de8321-9718-4294-9b3e-5a316a4e877a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.116223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48990be-01e0-4be7-b83f-a8d29b435633 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.123887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.124485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.124756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.125375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.126104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:40.126719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.128121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.140452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 296/598] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:40.141221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.156028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.156953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.157423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.158019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/910ca69bc9bf4f328d9537f0b9bb1658/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.174593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=910ca69bc9bf4f328d9537f0b9bb1658, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.176228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.191950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 303/599] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:40 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/910ca69bc9bf4f328d9537f0b9bb1658/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:40.193198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.270194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f1a899-3029-44b5-adf0-f5b601bfe949 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.307798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.318014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.318644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.318921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.319192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.323266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.331010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dfb6b4-5f57-4743-88e6-3c0cd286c3bb None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.336941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.338451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:40.358982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:40.360238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:40.361154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:40.361154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:40.361154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:40.361154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 297/600] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:40.361154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.374291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.375302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.375790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.376301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.377224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:40.377799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.379467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.397813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 304/601] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:40.397813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.484155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d9c38f-2741-4e19-b7d6-8b930b517bbd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.514289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb5e9-ce12-4b05-b30a-b4d2c787f919 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.530530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.531259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.531456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.531722 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/0f8ea3cf81f944a2b02e59b3ff049f6d/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.550236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=0f8ea3cf81f944a2b02e59b3ff049f6d, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.551923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.553623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.554639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.555224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.555706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.556598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:40.557553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.560012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.568337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 298/602] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:40 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/0f8ea3cf81f944a2b02e59b3ff049f6d/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:40.569833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.576090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 305/603] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:40.581615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.586698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.617762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:40.713811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad3d07-42da-40c8-a43d-0df11f797bab None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.747969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7501b5-10c4-427b-a35f-f9a7579b142b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:40.875140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.876635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.877188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.877688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/c21dea41dba24bf88466a7c3f00b8c2b/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.912896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:40.912896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.912896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.912896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:40.918995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=c21dea41dba24bf88466a7c3f00b8c2b, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.920778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.932309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:40.935685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:40.953846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 299/604] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:40 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/c21dea41dba24bf88466a7c3f00b8c2b/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-64c08624-c442-464a-9dae-2fab08dde295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:40.960517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:40.961765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:40.962944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:40.971201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:40.973347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 306/605] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:40.993895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbea667-47b8-47c3-b2ef-1674fa60fc9b None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:40.994324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbea667-47b8-47c3-b2ef-1674fa60fc9b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:40.994599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbea667-47b8-47c3-b2ef-1674fa60fc9b None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.001775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 307/606] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:41.004331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.004691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.005024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.049899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:41.051493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:41.195246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.215294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 308/607] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:41.236849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.253097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53603ce3-6b80-4bff-b513-c4670c21401b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.292176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.292879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.293227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.294147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.294972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:41.295346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.296871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.310019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 300/608] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:41.315535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.452778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082c42d-3a93-4cb2-862e-6bea95c2468f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.493659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.494442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.494794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.495134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/36fab0873915497cada62e8ea73bd4d6/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.501135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.517516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=36fab0873915497cada62e8ea73bd4d6, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.519076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.536771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 309/609] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:41 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/36fab0873915497cada62e8ea73bd4d6/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:41.662883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d736ff9-9a33-440a-ac5f-95b274bda24b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.715871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.716458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.716740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.717010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.733953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:41.739382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:41.740585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:41.741588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 301/610] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:41.748790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:41.824273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:41.912451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e8f42-585a-43e5-9059-7dcb40b17daa None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:41.962403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:41.963074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:41.963271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:41.963544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:41.964201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:41.964502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:41.970630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:41.978833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 310/611] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:41.984474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.005868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.141702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7ca42-3e53-4272-8f92-a076421f61c3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.175634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.176263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.176537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.176804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.188320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.189773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:42.204740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:42.206896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:42.208938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:42.208938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:42.208938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:42.208938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 302/612] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:42.214935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.354249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5a527d-ac32-47e7-b911-8cfb4e5b4f8b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.361160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.389190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.389831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.390112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.390384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.391016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:42.391318 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.392672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.404564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 311/613] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.409908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.501348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:42.560356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce639d8-997c-4b9a-a752-a6a8bae511c0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.619302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.620174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.620590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.621021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] PATH_INFO: `/v3/users/swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.629479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.631111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3477c9d6-29a9-4bae-bedc-2a73de8d965d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:42.646938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:42.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:42.649209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:42.651335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 303/614] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:42.657121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.814208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44192a3d-04db-47d7-986f-7eb6f2553169 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:42.865877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:42.866494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.866775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.867043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.867761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:42.868061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:42.869532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:42.885272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29017-90e5-4833-9ff2-807a0c272259 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.885923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29017-90e5-4833-9ff2-807a0c272259 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.886393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db29017-90e5-4833-9ff2-807a0c272259 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.888185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 304/615] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:42.891402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 312/616] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:42.893802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:42.894293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:42.894754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:42.896799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:42.916293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:42.917838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:42.944379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.051046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 305/617] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:43.084572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44352c6b-716b-4939-af4f-b50a10f8ec61 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.133016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.133576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.133890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.134167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.147697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.149218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:43.169551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:43.170551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:43.171982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:43.171982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:43.171982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:43.171982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 313/618] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:43.176311 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.243102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.307613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4617c-5966-4628-a3a6-e761eca53552 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.359396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.359999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.360269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.360538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.361181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:43.361477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.362873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.375795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 306/619] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:43.384102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.419579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.492339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.519614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab177721-9817-475c-bd2f-b4ccfd5b7fef None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.539437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadd849c-3777-4217-9af3-bb55c1ba9fb0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.550587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.560167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.561210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.561696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.562308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.570261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.572852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.584522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57d047-55c6-4178-af9a-a587a1677ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.585151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57d047-55c6-4178-af9a-a587a1677ac1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.585428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57d047-55c6-4178-af9a-a587a1677ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.585701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57d047-55c6-4178-af9a-a587a1677ac1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ef5edf86-08e2-4a46-8509-a32078706f1c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:43.589124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:43.590084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:43.590984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 314/620] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:43.592058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bab1fd-f15f-47c3-814d-e578c985c2a3 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.592173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bab1fd-f15f-47c3-814d-e578c985c2a3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.597753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bab1fd-f15f-47c3-814d-e578c985c2a3 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.597753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 315/621] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:43.599396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bab1fd-f15f-47c3-814d-e578c985c2a3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.606593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57d047-55c6-4178-af9a-a587a1677ac1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-cc57d047-55c6-4178-af9a-a587a1677ac1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:43.614506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:43.615506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:43.616338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 307/622] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:43.618346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.618346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.619030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.644664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:43.646142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:43.719149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 308/623] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:43.720243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.738223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bab1fd-f15f-47c3-814d-e578c985c2a3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.800957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.801775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.802155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.802568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.803365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:43.803761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.805341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.812055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.843727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 316/624] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:43.845165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12bfbda-f457-4cdd-8450-d22f3913dab2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:43.849551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:43.854130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:43.915079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:43.915599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.915871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.916140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.916802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:43.917099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:43.919273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc9de58-a7bf-42ea-b5bb-6b816e006630 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:43.927955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 309/625] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:43.934744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f73e7-59c0-4fb2-b3b0-dc3c8adf7929 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:43.935026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f73e7-59c0-4fb2-b3b0-dc3c8adf7929 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:43.935299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f73e7-59c0-4fb2-b3b0-dc3c8adf7929 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:43.937858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 310/626] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:43.937858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f73e7-59c0-4fb2-b3b0-dc3c8adf7929 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.000875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e17101-de5b-4fa5-a15f-0e6d1fcd2153 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.036131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4c87bb-c1c5-4081-872c-3abd882b9215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.036796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c87bb-c1c5-4081-872c-3abd882b9215 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.037074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c87bb-c1c5-4081-872c-3abd882b9215 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.037348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4c87bb-c1c5-4081-872c-3abd882b9215 None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/38d08c9aa2604871bec462424708b85b/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.052783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c87bb-c1c5-4081-872c-3abd882b9215 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=38d08c9aa2604871bec462424708b85b, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.054319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4c87bb-c1c5-4081-872c-3abd882b9215 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:44.076699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 317/627] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:43 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/38d08c9aa2604871bec462424708b85b/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:44.076699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.076699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.076699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.093208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f73e7-59c0-4fb2-b3b0-dc3c8adf7929 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.094238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:44.095601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:44.132339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.133261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.133602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.134014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.149596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:44.150099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.151600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:44.169622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:44.170674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:44.171568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:44.171568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:44.171568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:44.171568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 311/628] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:44.172124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.194231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:44.195190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 318/629] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:44.196213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.341657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c94ed2-1be9-4e5e-83b7-27ef1ce65569 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.356442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b7c6d4-6f7f-40b5-bdf4-1d090b8a3bcf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.388620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.389229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.389499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.389791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.390422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:44.390721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.392085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc47edc-5d9c-4d47-ac89-6c606614d34e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:44.404501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 319/630] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:44.405656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1443c-38a7-4c66-b3ac-233bb89685b2 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.405961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1443c-38a7-4c66-b3ac-233bb89685b2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.406235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1443c-38a7-4c66-b3ac-233bb89685b2 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.407432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abf094f-ab1f-42c2-989b-51c711c51ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.407684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 320/631] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:44.408471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf094f-ab1f-42c2-989b-51c711c51ea6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.408745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf094f-ab1f-42c2-989b-51c711c51ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.409014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abf094f-ab1f-42c2-989b-51c711c51ea6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.409853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1443c-38a7-4c66-b3ac-233bb89685b2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.424101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abf094f-ab1f-42c2-989b-51c711c51ea6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-9abf094f-ab1f-42c2-989b-51c711c51ea6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:44.429196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:44.430120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:44.430960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 312/632] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:44.432365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.432365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.432529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.450247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:44.451841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:44.456116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:44.522696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 313/633] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:44.530081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.534576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:44.545005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1443c-38a7-4c66-b3ac-233bb89685b2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.580049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.580672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.580948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.581219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.589096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.591109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:44.602536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:44.603512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:44.604490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:44.608820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 321/634] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:44.609637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.655493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81872a81-c306-42aa-bd78-e62bb18062da None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.700805 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.701987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.702461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.702913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.703754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:44.704222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.705753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:44.716088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 314/635] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:44.717198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.745123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:44.754762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a288-d060-4210-aadf-a1fdaf2dc668 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.788457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:44.804446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.805087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.805365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.805633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.816584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:44.821942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:44.823055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 322/636] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:44.824654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.836565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ed29f3-0cf3-4290-9595-329926bbeda8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.870413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.871052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.871740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.872010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.872728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:44.873023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.875188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:44.892649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 315/637] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:44.893448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:44.940664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae76a6f-6b43-46e3-8d5b-41833f671258 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:44.978169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:44.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:44.979027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:44.979296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:44.990127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:44.991540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.002463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810229a-dd1c-4cb6-b638-01c777e03537 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:45.006341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.007284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:45.008175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:45.008175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.008175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:45.012549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 323/638] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.013373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.024531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.028388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.040193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.040818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.041091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.041360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.051935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:45.056990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.058792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 316/639] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.059653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.126104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36513f-6525-4120-a9c5-39210f0fade0 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.162102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.162703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.162979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.163247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.163896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.164193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.165556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.167211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0540e694-3419-40e7-997c-a8bba5b78880 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.178464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 324/640] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.178464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.208951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.209566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.209839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.210108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.246638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.268234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.278790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c3788d-f552-4161-b6b7-f06c090d1c22 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.285474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.287018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.313171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.313789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.314184 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.314457 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.315205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.315504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.316993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.333698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 325/641] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.333698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.401086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 317/642] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:45.402223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.440312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d89f34f-8cf2-45a4-9533-ab422bc79e4f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.470953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.478207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.478926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.479258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.479539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.480720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.481022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.482401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.490450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 326/643] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.491240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.503922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.521792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c40fcd-ca3d-400c-82b1-2c71916eb34d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.557433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.558080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.558987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.559267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.569978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.571416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:45.586292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.587354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:45.588789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:45.588789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.588789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:45.588789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 318/644] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.588789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.600253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f0a2e-4c9f-443c-8d58-67c29f065589 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.635826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.636480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.636757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.637027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.644433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.645874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:45.656768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.657918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 327/645] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.659219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.705466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1af7e8-ca38-46b4-9391-a8e6980c6350 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.720220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.744894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.745516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.746409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.746877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.749408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.758629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.759992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.771814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07500ab-6ddc-4348-a2cd-814910507d97 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:45.774709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.775633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:45.777551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:45.777551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.777551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:45.777551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 319/646] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.777551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.814973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.815616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.815900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.816177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.828077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.829600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:45.844641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:45.845624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:45.846545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:45.846545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:45.846545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:45.846545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 328/647] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:45.846545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.894156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613fc84e-0026-444d-becd-2bc58c628c94 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.928568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.929236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.929512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.929806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:45.930444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:45.930741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:45.932333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:45.944302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 320/648] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:45.944897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:45.959443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1598c8-c661-4189-8c70-5ab20be39edd None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:45.962244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.996344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:45.998641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:45.999271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:45.999551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:45.999867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.000893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.001203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.002670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.024329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 329/649] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.024329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.051225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b382-fbc2-484d-a7c9-a1fcacfdbec4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.085382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.086015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.086294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.086566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.087200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.087498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.088834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.100974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 321/650] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.101768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.130770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4109f6-c24a-4e46-9a15-0e7914c8ae29 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.165499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.166150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.166426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.166694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.167354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.167686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.169033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.180316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 330/651] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.181354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.208955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.219707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4ee79f-a1db-4364-986d-55ac60272890 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.229264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.255548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.256176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.256447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.256715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] PATH_INFO: `/v3/users/glance` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.265849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.267319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:46.281131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.282763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 322/652] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:46.285030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.299149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa71d4-4168-4b8d-8ffe-e603ca1dd8f3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.341149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.342192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.343229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.343704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.394696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e60e43-829a-434d-a0b5-82e58cc918ec None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.427972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.428299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.428913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.429183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.429429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.429532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.436196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.437588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:46.448388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.449808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 323/653] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:46.450714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.457872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.479293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.532814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 331/654] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:46.533723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.549263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8c9f95-35d2-48e4-b47a-1b0f79d69641 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.582944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.583559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.583844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.584108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.591697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.593104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-9f2fb68a-0a8a-4964-814d-5264b839f355 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:46.604013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.605135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:46.606101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 324/655] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:46.606707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a853-eed8-45dd-a2e1-a803900d2993 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.606775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a853-eed8-45dd-a2e1-a803900d2993 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.607045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35a853-eed8-45dd-a2e1-a803900d2993 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.609758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 325/656] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:46.609758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a853-eed8-45dd-a2e1-a803900d2993 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.635173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc4a0b-97a1-41b6-85f7-917afcdc4d1b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.657411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.669082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.669675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.669977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.670251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.670965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.671263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.672710 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.689566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 332/657] 10.222.0.15 () {56 vars in 1137 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.690368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.698025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.711612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35a853-eed8-45dd-a2e1-a803900d2993 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.744445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.745043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.745320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.746128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.746812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.747112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.748535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.760019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 326/658] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.760788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.789804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd57d0-211a-4c5d-b8ae-ba51ff4ef391 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.822943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.823618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.824413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.824682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.834555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.836008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-89a77073-70fa-4d0d-8844-ff4c4d78248f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:46.851943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:46.853038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:46.854142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:46.854142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:46.854142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:46.854142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 333/659] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:46.854433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.854515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.854789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.863343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0c2e2-2263-40ff-a636-72fd5b49d4d9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:46.873073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:46.874294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:46.896995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:46.897584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:46.897895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:46.898158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:46.898897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:46.899198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:46.899900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.900675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:46.918049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 327/660] 10.222.0.15 () {56 vars in 1139 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:46.918049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:46.927050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:46.946227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 334/661] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:46.947058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.026266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5eda21-3dbb-4fbb-9557-5b8b05a63ef2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.047625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3dfc4-7947-4264-829c-1f832e792b6e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.063249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.063849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.064238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.064508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.081135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.081751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.082047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.082319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.083034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=910ca69bc9bf4f328d9537f0b9bb1658) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.083339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.084759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.097227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 335/662] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/role_assignments?user.id=910ca69bc9bf4f328d9537f0b9bb1658&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.097227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.130062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.155472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.157157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.166335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.229668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866bc51-ca5d-437d-8fa1-68fcc0ac0aac None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.267061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.267711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.267991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.268261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.268901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.269196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.270619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.277917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 328/663] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:47.277917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.289347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 336/664] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.290139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.382726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.399523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76673c86-a4a0-499f-8571-d3b96e1eb982 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.403585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adece83-ce78-4b46-b9fe-39a18cd86572 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.411834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.434781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.435523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.435879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.436160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.444098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.444975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.445461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.445825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.446523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:47.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:47.452628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 337/665] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:47.458503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.528336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.529791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.562007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c794e23-9fbb-4211-8c3f-8871bfb5a5ac None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.601449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.602128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.602405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.602675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.614819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.616282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.623005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:47.631719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:47.632663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:47.633592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:47.633592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:47.633592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:47.633592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 338/666] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:47.633592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.656339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 329/667] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:47.657181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.667572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.748687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ef7f8c-320d-418d-a552-f58601f07ed3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.767039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8651ded-6999-41cb-b56b-f4ef9a77550d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.784061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.784681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.784955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.785221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.792813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.794290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.802931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.803524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.803794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.804058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.804824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.805122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:47.806002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:47.806848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:47.807789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:47.808259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.808329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 339/668] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:47.808396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.825288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 330/669] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.825288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.882085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.900780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:47.931965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c45be-46ae-4b25-9907-6ef91da5c4c0 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.937806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18f197-e88c-46dc-a119-654da6b051d9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:47.972086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.972697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.973072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.973316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.973925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:47.974233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:47.975605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:47.982647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:47.983301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:47.983554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:47.983816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:47.988582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 340/670] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:47.989213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:47.999892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.001695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:48.017279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:48.018376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:48.019254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:48.019254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:48.019254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:48.019254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 331/671] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:48.019254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.100078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ff10e4-c965-42bf-a938-9ab11a4a829a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.118065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.126645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a780b964-946b-40d9-9e39-cd9cd5d23515 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.137828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.138440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.138719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.138994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.139675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.139985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.141404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.147521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.154200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 341/672] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.155271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.162655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.163282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.163557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.163828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.174616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.176033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:48.190850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:48.191987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:48.192892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:48.192892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:48.192892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:48.192892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 332/673] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:48.192892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.260665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b10dd2-e68e-413e-ae44-7565154c9131 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.297254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b411f-1873-47f6-9e0b-d77e58366439 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.297889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.298492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.298766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.299023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.306285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.308226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:48.319567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:48.320488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 342/674] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:48.321986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.337361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.337993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.338446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.338720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.339773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.340349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.341706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.359634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 333/675] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.360454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.363437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.396905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.432230 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef08e8-1531-4176-bd31-47ddd4650cf5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.468505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.469200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.469577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.469894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.470605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=0f8ea3cf81f944a2b02e59b3ff049f6d) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.470908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.472312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.472962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab090106-0a78-4700-9d28-373830f0c5be None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.485905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 343/676] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/role_assignments?user.id=0f8ea3cf81f944a2b02e59b3ff049f6d&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.485905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.518760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.519376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.520067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.520336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.521276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.521578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.523625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.542926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 334/677] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.544048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.603137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423776c-433e-4470-a82b-9323cad62859 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.605304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.644325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.645010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.645287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.645601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.646321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.646622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.648212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.652346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.663520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c71f-6599-484f-90c7-899f12265eef None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.664253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 344/678] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.664899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.710882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.712315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.712819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.713308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.721973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.723725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:48.737008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:48.737963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:48.738968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 335/679] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:48.739467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.787643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103f9c4-b87a-461c-a8cd-1d8c3e6a9174 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.834231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.835438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.835894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.836586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.848519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.850773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.858461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06df9a75-9e21-4646-a09c-fc3a575ebd22 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:48.867120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:48.868059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:48.869085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:48.871676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 345/680] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:48.873093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.906556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:48.907301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:48.907672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:48.908197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:48.909017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=c21dea41dba24bf88466a7c3f00b8c2b) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:48.909435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:48.911189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:48.916412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:48.920601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 336/681] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/role_assignments?user.id=c21dea41dba24bf88466a7c3f00b8c2b&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:48.921606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:48.945976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.088421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62d2f31-c7fd-402e-a8ad-1edfab857986 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.134113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a54f39-29c7-48a9-9fde-da3ecb5fd1b6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.156075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.156690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.156966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.157236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.157982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.158286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.159716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.180107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 346/682] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:49.181514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.182201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.182478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.182786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.183477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.183586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.183798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.185334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.208859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 337/683] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:49.218901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.283035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.307692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f668d0-4f70-4b21-89fb-82e43c451b03 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.314113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.342985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a49f225-317d-4de4-97f5-8fdeef6cc0ee None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.359416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.360451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.360870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.361144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.362266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=36fab0873915497cada62e8ea73bd4d6) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:49.362266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.363696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ac189b-22fb-4a79-b880-e4ce1b4fffbb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.370556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 347/684] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/role_assignments?user.id=36fab0873915497cada62e8ea73bd4d6&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:49.371691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612df69-1816-4c3c-8038-3f74ee5ab2e2 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.371970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612df69-1816-4c3c-8038-3f74ee5ab2e2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.372240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7612df69-1816-4c3c-8038-3f74ee5ab2e2 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.377691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 348/685] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:49.379415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.379709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.379965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.407878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:49.409701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:49.418496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.419676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.419676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.419877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.552877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 349/686] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:49.561648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.563246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.643915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.669956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 338/687] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 19:37:49.678841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.771460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:49.795563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362754-38b0-4792-ab2e-f5b7bf8f8616 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:49.833524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:49.833524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:49.833524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:49.833524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:49.843528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:49.844969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:49.859545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:49.860506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:49.861557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:49.861557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:49.861557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:49.861557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 350/688] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:49.867634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.882060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:49.980383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46fa9b1-7fb3-4742-9897-8064cf018255 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.025467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.026139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.026423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.026693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.027445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.027751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.029226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.044083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034f70a-83e7-4736-9a93-dfc2e33031a1 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.056481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 339/689] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:50.063323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.072311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.080748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.081384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.081664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.081969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.092725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:50.098191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:50.099141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:50.099985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 351/690] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:50.105746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.129517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.216451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6d33c4-fae7-4b2e-9525-e628fd9cd6c3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.239251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9fd66e-3d22-4079-af95-db5cb8479255 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.252094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.252704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.252978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.253250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.253938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.254240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.255648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.272454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 352/691] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:50.278573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.287307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.287934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.288205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.288470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.302744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.304162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:50.321188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:50.324048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:50.326161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 340/692] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:50.331099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.399167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.424212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.431232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae2bcd-7ea8-4e16-a2b8-9565620a2940 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.471760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.472364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.472641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.472914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.473606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.473941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.475366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.497221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995136f-9c11-46ee-94da-075a53e5a09c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.504323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 341/693] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:50.505640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.567478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.568121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.568401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.568673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.592263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.593712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.612611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e754a-e965-48b1-a120-adf66c0f783f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:50.618460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:50.619382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:50.620322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:50.620322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:50.620322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:50.620322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 353/694] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:50.625489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.649713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.650342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.650621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.650893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.652192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=38d08c9aa2604871bec462424708b85b) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.652497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.653982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.665940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 342/695] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/role_assignments?user.id=38d08c9aa2604871bec462424708b85b&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:50.801576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5baae1-10c2-4334-81c6-0f602d62e560 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:50.803346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.861249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:50.861897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:50.862178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:50.862450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:50.863163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:50.863465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:50.864877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:50.877671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 354/696] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:50.883134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:50.903165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:50.990071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf619c05-68b6-45aa-b2a8-5602ce1353d7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.025491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.026229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.026556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.026878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] PATH_INFO: `/v3/users/nova` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.034335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.035757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-c914a8bc-d592-464b-8c90-a42fe6f7eaee None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:51.048114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:51.049114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:51.050105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 343/697] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:51.055304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.165837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943de8d9-48e6-4f03-8d02-b1f015433150 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.203309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.203925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.204200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.204469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.205192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:51.205492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.207076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.225940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 355/698] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:51.229792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ea5c4-b32d-4360-a6c8-2510e8ca81d7 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.230539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ea5c4-b32d-4360-a6c8-2510e8ca81d7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.231009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872ea5c4-b32d-4360-a6c8-2510e8ca81d7 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.231350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.233081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 344/699] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:51.238592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.239118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.239638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.255450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.263307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:51.264617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:51.341966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dfe47c-3ee8-4886-8f63-17c0015b8fb6 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.342834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 345/700] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:51.380201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.381053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.381524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.381991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.457418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.459690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.486325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.594245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 356/701] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:37:51.599936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.616103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.659188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.749009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d983d92-bdce-4498-8491-505b60ff3ce6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.808485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.809615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.810510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.810930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.816113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4376d5-346a-4b77-a93e-c2444d3767d1 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:51.824704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.826978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.845981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-d9f1c02f-7d51-4ab3-8dc2-641bc56d08f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:51.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:51.849412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:51.850344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:51.850344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:51.850344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:51.854816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 346/702] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:51.856393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a79db-da09-4877-a182-87aed1096b51 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.856805 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a79db-da09-4877-a182-87aed1096b51 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.857222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a79db-da09-4877-a182-87aed1096b51 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.859156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 347/703] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:51.859991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a79db-da09-4877-a182-87aed1096b51 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:51.872689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:51.873458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.875510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.875510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.875510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:51.876307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c17016bc-7e2c-407b-803f-c31a02e32efc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:51.889047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:51.890321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:51.891368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:51.898456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 357/704] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:51.898456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:51.898456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:51.898456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:51.917372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:51.918783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:51.942337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:51.986848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a79db-da09-4877-a182-87aed1096b51 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.017875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 358/705] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:52.019033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.033671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.034582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.035061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.035506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.036320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.036793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.038359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.056830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 348/706] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.063112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.210636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88faa337-86c4-441d-a036-bd3d0859e976 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.235263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.244756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4643c0c-76e6-4898-82de-468607e9a048 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.260692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.261332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.261608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.262734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.263401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.263707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.265110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e33a1-46ad-400f-a57b-8ae5738280a1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.270794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 359/707] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.272069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcb0c-53f0-4c09-a8d4-d7b696ecf7a2 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.272350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcb0c-53f0-4c09-a8d4-d7b696ecf7a2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.272622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbfcb0c-53f0-4c09-a8d4-d7b696ecf7a2 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.281997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 360/708] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:52.281997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfcb0c-53f0-4c09-a8d4-d7b696ecf7a2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.282725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.307036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.307670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.307942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.308203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.318164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.320553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-8cdf611f-7f40-4e66-a7c4-c732ae90844e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:52.336628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:52.337596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:52.339184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:52.340288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 349/709] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:52.345082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349ade5-c824-470e-9200-caeacf571838 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.345082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349ade5-c824-470e-9200-caeacf571838 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.345082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349ade5-c824-470e-9200-caeacf571838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.359201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2349ade5-c824-470e-9200-caeacf571838 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:52.360365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2349ade5-c824-470e-9200-caeacf571838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:52.400459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbfcb0c-53f0-4c09-a8d4-d7b696ecf7a2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.437754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 350/710] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:52.437754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349ade5-c824-470e-9200-caeacf571838 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.446356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.447061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.447399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.448363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.449057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.449468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.451045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.462242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 361/711] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:52.463908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.558581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.606564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349ade5-c824-470e-9200-caeacf571838 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.629837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.661470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd02c5-a8b5-409f-962e-4869a860e993 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.664725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.665426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.665822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.666159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.677603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.679078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:52.697050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:52.698651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:52.699550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:52.699550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:52.699550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:52.700208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 351/712] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:52.707545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.708159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.708430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.708698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.709379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.709677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.711110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.714754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.726403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 362/713] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.732453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.865432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8e1a2-eb87-414f-98c1-e8f42a0b5ce6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.888967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daa27a-b484-4a48-826f-cb0193f3fe1a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:52.890548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:52.903461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.904057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.904330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.904600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.905229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.905525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.907530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.921193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 352/714] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.925951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.933177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:52.933993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:52.934442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:52.934859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:52.935776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=910ca69bc9bf4f328d9537f0b9bb1658) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:52.936241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:52.937939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:52.949931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 363/715] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/role_assignments?user.id=910ca69bc9bf4f328d9537f0b9bb1658&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:52.949931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:52.963210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.113320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f40d1c-c604-4635-aee5-36dec427608d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.158745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094d647c-1855-4517-ade1-15b1a7a85556 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.160677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.161685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.162207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.162689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] PATH_INFO: `/v3/services/network` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.163524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.165090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:53.178999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:53.180335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:53.181465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 353/716] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:52 2022] GET /identity/v3/services/network => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:53.186766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:53.202147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.203098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.203584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.204101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.204951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.206612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.234889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 364/717] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:37:52 2022] POST /identity/v3/users => generated 254 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:53.293515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.317108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f2256-8eda-4113-a0c8-b73555cf149d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.365131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.365765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.366000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.366268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.367028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] RBAC: Adding query filter params (name=network) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:53.367325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.368695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dc109f-22d8-45a4-a0d6-a1d028318a22 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.374213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 354/718] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:53.379949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:53.416085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:53.538312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266784c-32dd-4af6-857e-7f22e65c4bd0 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:53.581000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:53.582029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.582524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.583000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.583855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] RBAC: Adding query filter params (type=network) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:53.584362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:53.585962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a2338-6705-4aef-8c88-bb82ded35679 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:53.592125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 365/719] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:37:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:53.606647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a537431-fc4a-44d4-b953-f41384da7e16 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.606996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a537431-fc4a-44d4-b953-f41384da7e16 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.607199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a537431-fc4a-44d4-b953-f41384da7e16 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.608885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 355/720] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:53.614933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:53.615300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:53.615487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:53.635763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:53.637014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:53.721108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 366/721] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:54.033509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a537431-fc4a-44d4-b953-f41384da7e16 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.151403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a537431-fc4a-44d4-b953-f41384da7e16 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:54.199507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:54.200145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.200422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.200689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.215491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:54.220739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:54.221675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:54.222590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 356/722] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:54.227659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.263540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:54.366448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852ff30-178e-4f2f-bed9-4b7cc672ec07 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:54.415314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:54.415933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.416745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.417015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.417674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:54.417994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:54.419369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:54.434139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 367/723] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:54.440022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.457981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:54.554138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fb5832-6cfa-4c39-b1a7-e28441f07cfb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:54.605717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:54.606722 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.607227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.607694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.620613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:54.622464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:54.648254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:54.649369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:54.650342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:54.650342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:54.650342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:54.650342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 357/724] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:54.655106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.758250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:54.782927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ff95a2-19d9-4c06-9dd7-62d8ad0aa261 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:54.825160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:54.825821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:54.828932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:54.828932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:54.828932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:54.828932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:54.828932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:54.845883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 368/725] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:54.851140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:54.950572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.012080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73de0498-c351-41bd-9488-8e4f719b3199 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.060078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.060706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.060987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.061254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.075889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.077315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-c24e7228-99c2-4c5b-9049-3a08e8e38a24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:55.093652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:55.094712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:55.095654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 358/726] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:55.100716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.109011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274fbaf-4742-49a5-8bc3-2e00729fceda None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.109513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274fbaf-4742-49a5-8bc3-2e00729fceda None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.109988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b274fbaf-4742-49a5-8bc3-2e00729fceda None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.111702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 359/727] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:37:55.117139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.117655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.118199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.143496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:55.145016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:55.211702 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c735-c52a-4256-a91f-79540e34a793 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.263131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.263497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 360/728] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:55.263703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.264003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.264264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.264993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:55.265299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.273808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.276132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.286926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 369/729] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:55.292439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.373646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.441672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dde185-4053-4a88-bb01-b7dc783b5505 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.490218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528d772c-3cd8-4110-8799-0c90ee9fac46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.490828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d772c-3cd8-4110-8799-0c90ee9fac46 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.491104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d772c-3cd8-4110-8799-0c90ee9fac46 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.491375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528d772c-3cd8-4110-8799-0c90ee9fac46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.569033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.612396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d772c-3cd8-4110-8799-0c90ee9fac46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.613870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528d772c-3cd8-4110-8799-0c90ee9fac46 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.706526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.727633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8e355-be14-4d7c-994f-ea0e661158e5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.767210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ffe85-30c9-4b02-9ee0-6e783c7ccedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.767734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ffe85-30c9-4b02-9ee0-6e783c7ccedc None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.768013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ffe85-30c9-4b02-9ee0-6e783c7ccedc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.768288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ffe85-30c9-4b02-9ee0-6e783c7ccedc None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.768949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ffe85-30c9-4b02-9ee0-6e783c7ccedc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.770364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ffe85-30c9-4b02-9ee0-6e783c7ccedc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.774977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 361/730] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 19:37:55.777197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289108-89b5-4f86-a6e7-972a3a0b00d9 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.777929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289108-89b5-4f86-a6e7-972a3a0b00d9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.778406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b289108-89b5-4f86-a6e7-972a3a0b00d9 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.780510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 362/731] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:55.781393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b289108-89b5-4f86-a6e7-972a3a0b00d9 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:55.791079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 370/732] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/services => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:55.792545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.792831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.793164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.814150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:55.815331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:55.873414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:55.900667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 371/733] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:55 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:55.910365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b289108-89b5-4f86-a6e7-972a3a0b00d9 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:55.950345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:55.951451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:55.951959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:55.952429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:55.964761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:55.966659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:55.983397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:55.984597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:55.985609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:55.985609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:55.985609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:55.988177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 363/734] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:55.993967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.104600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db3d773-a0da-4a77-a425-d5c2812fdefd None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.147707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.148331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.148670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.148950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.149609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.149964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.151380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.163727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 372/735] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.169714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.178558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.229264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.310264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.344520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70878890-aeba-4224-974e-bbd595be58e3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.372178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a79b52-d68c-4d54-963f-1d41c54793b2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.390866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.391495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.391767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.392034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.402421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.404657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.407765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.408400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.408678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.408947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-509809f9-cb8f-444a-817a-66962a85d292 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:56.419117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:56.420134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:56.421028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:56.421449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:56.421521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 364/736] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:56.422089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a878f-fc3d-4871-9373-edeec898786d None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.422256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a878f-fc3d-4871-9373-edeec898786d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.425069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a878f-fc3d-4871-9373-edeec898786d None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:56.426252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:56.427170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:37:56.428912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:56.430729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 365/738] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:56.430729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.430729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.430729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.434254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 373/738] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:56.434254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.451497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:56.452942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:56.532066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.549607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 366/739] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 19:37:56.550618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.558091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0409f7-f168-4ecb-990f-b2b8bbb1a070 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.560485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.618867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.619579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.619856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.620126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.620813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.621114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.622634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302c88a-1dff-44fb-acaa-934107f8ca1c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.635934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 374/740] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.637255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eab0b-6259-4136-9958-8653056b5195 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.637540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eab0b-6259-4136-9958-8653056b5195 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.637844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6eab0b-6259-4136-9958-8653056b5195 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.643149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 375/741] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:56.643796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6eab0b-6259-4136-9958-8653056b5195 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.683102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80704f46-77e0-46e9-b70d-30dfe30faf2d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.723133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.724059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.724535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.725033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.726102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.726407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.727791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bededaa4-3e95-4622-8f3d-858ec4d2c0bb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.736816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 367/742] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.738560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.738560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.738849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.760995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6eab0b-6259-4136-9958-8653056b5195 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:56.765246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:56.766671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:56.805643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:56.806320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:56.806603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:56.806887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:56.807592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=38d08c9aa2604871bec462424708b85b) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:56.807889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:56.809305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:56.821809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 376/743] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/role_assignments?user.id=38d08c9aa2604871bec462424708b85b&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:56.823588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:56.850380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.872040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:56.872238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 368/744] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:56 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 19:37:56.874022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.034416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fd4f9-1c91-4cf7-af09-704c10250c0c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.054994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8740d44f-0b43-4153-8083-58ba3852e46c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.096349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.096984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.097264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.097535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.114350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.114950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.115225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.115489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.116139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.117482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.121913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.123331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:57.141178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:57.142204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:57.143394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:57.143394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:57.143394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:57.143394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 377/745] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:57.151492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 369/746] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:37:56 2022] POST /identity/v3/services => generated 240 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:37:57.152465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.214167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.270460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.272781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.318020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7b9ef1-2ebc-4d59-b3c7-faf898180318 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.354429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b3724-8b61-4f13-b8a7-0370ddb9c546 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.378955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.379585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.379861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.380132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.380791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:57.381089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.382547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.404893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 378/747] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:57.410274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.427886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.428522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.428795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.429061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.446377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:37:57.452252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:57.453644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:37:57.454620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:57.458128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 370/748] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:57.465312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.544666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a540b3-b131-4ebb-8188-2fe8b4358ac7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.580975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.582000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.582476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.582941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.589197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fdc25e-df78-4863-b523-d58aa36a5912 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.591042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.591907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.593560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.595758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:57.608793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:57.609812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:37:57.610786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:57.613383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 379/749] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:57.619118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.641443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.641963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.642240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.642509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.643181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:57.643484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.644847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.654677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 371/750] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:57.661633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.796067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a213552-e617-4280-8d87-50f5b2449455 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.840342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.841241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.841747 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.842326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:57.843259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:57.843987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:57.845618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:57.866563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 380/751] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:57.872939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:57.895869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5879dc6e-8c41-4cc5-ab7a-9a73c11fd7eb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.938056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:57.989063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7d250-cbb2-4260-8727-5135efe946d9 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:57.991002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:57.991646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:57.991923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:57.992195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.019998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.021462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:58.044405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:58.045421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:58.046337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:58.046337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:58.046337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:58.049358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 372/752] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:58.058283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.058283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.058283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.058283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.064307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.081277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.082778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.094270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:58.105000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:58.106043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:58.106923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:58.106923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:58.106923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:58.107541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 381/753] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:58.114809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.267074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0708b1-2873-4f41-945f-0e76897102df None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.279730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca753c24-b2b8-419a-9cd4-d7cb103cd464 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.290103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.310426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.311026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.312497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.312769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.313408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:58.313702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.316634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.340549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 373/754] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:58.346574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.347238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.347516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.347785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.348425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:58.348725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.350181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.353956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.377540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 382/755] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:58.386855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.471437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.500221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50289f2-87cf-4210-a86e-1a023df73517 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.546269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.547115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.547396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.547933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.552810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf031ee7-35d7-4d95-9db0-bd5e038c25ba None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.557007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.558479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-1bf91016-114b-446a-bc3f-4ce2fcc8b3d2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:37:58.573291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:58.574379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:37:58.575330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:58.577860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 374/756] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:37:58.579185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa801d-6f07-43fb-a889-8c85a4119f62 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.579569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa801d-6f07-43fb-a889-8c85a4119f62 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.579843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa801d-6f07-43fb-a889-8c85a4119f62 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.581214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 375/757] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:58.583361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa801d-6f07-43fb-a889-8c85a4119f62 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.598913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.599545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.599824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.600096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.607767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.609204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-33b6995c-e764-4640-b390-0cd290c8b1cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:37:58.620666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:58.621558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:58.622511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 383/758] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:58.623572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.623987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.624272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.642512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:58.643762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:58.670344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.720452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 384/759] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:58 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:58.721227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.728928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9aa801d-6f07-43fb-a889-8c85a4119f62 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.785310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.786538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.786976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.787382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.788267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:58.788699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.790321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.793984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:58.810529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 376/760] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:58.818646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.861365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c67c54-7723-4cdd-bc10-7bcbc5ec5ce5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:58.897045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:58.897842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:58.898225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:58.898611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:58.899465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:58.900475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:58.902145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:58.914730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 385/761] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:58.919999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:58.951718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.020490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290ede3-5f01-4b48-b2e1-cf32f2c4b31f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.057779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b04507f-2daa-45d2-93b4-304086df04a4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.082851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.083183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.083350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.083626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.098839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.099505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.099788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.100062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/910ca69bc9bf4f328d9537f0b9bb1658/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.102315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=910ca69bc9bf4f328d9537f0b9bb1658, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.103790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.123249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 386/762] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:37:58 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/910ca69bc9bf4f328d9537f0b9bb1658/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:37:59.124376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.157803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.168693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.170923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.224305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.252657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd5217-1462-4577-b3de-4f4b33cdecd6 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.289038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.289698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.290020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.290317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.290894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.292280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:59.301474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:59.302603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:37:59.304101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 387/763] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:59.309317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.312381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 377/764] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:37:58 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:37:59.318805 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.470140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47394a02-4575-4e7a-bc32-b0d798717dbc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.489058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.498161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c51ef9-5002-4cf1-b187-e3fde6e4b145 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.513376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.514353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.514938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.515440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.516276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:59.516781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.518607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14991d81-1efe-45a5-a16e-4f50bb8803bf None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.524233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 388/765] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:37:59.525917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575e745-3b4b-4a4a-87d9-b6f466b626dc None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.526432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575e745-3b4b-4a4a-87d9-b6f466b626dc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.526925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0575e745-3b4b-4a4a-87d9-b6f466b626dc None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.533067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 389/766] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:37:59.534266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575e745-3b4b-4a4a-87d9-b6f466b626dc None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.543004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.543638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.543913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.544182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.545490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.561354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.563370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0e82a69d-28f5-4a53-a6f5-d9ddcfc66467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:37:59.580927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:37:59.581984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:37:59.583005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:37:59.583005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:37:59.583005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:37:59.585308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 378/767] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:37:59.586972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.587514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.588001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.607980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:37:59.609608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:37:59.661294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0575e745-3b4b-4a4a-87d9-b6f466b626dc None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.700062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 379/768] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:37:59 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:37:59.701274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.710447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.711072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.711356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.711629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.712311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:59.712614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.714005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.728868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 390/769] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:59.734005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.793602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.820315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaf8de8-182e-4dbb-ba25-9ef4c9504d87 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.887008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:37:59.887532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.887664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.887914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.892828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.893552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:59.893888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.895337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.918356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 380/770] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:37:59.919999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:37:59.942782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4b34d-d432-4f85-84cc-982e7a680b2a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:37:59.982435 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:37:59.983180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:37:59.983403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:37:59.983728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:37:59.984514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] RBAC: Adding query filter params (interface=public, service_id=80c9eed50cdb49f2947ade8463843a35, region_id=RegionOne) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:37:59.984929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:37:59.986322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12defa-c518-4f6d-8b28-e1ef006af6c1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:37:59.995093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 391/771] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/endpoints?service_id=80c9eed50cdb49f2947ade8463843a35&interface=public®ion_id=RegionOne => generated 194 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:00.000403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a41038-2d0e-4484-9774-446088329495 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.000403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a41038-2d0e-4484-9774-446088329495 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.000403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a41038-2d0e-4484-9774-446088329495 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.002329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 392/772] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:00.003127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a41038-2d0e-4484-9774-446088329495 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.066247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ace77-8ee4-4f76-8df3-0388fab4166b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.109189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.109840 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.110285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.110558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.114090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.119027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.120467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a41038-2d0e-4484-9774-446088329495 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.121158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-4ab505a0-962a-4c82-a0f7-ca4b35d455c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:00.135088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:00.136005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:00.136912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 381/773] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:00.137429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.138140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.138140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.156572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:00.157786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:00.161053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.161529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.161838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.162113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.162712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.164056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.166221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.178206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 393/774] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.179806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.244490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 382/775] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:00.245565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.301292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9e839b-7f54-4fc2-9dd1-287d7da8696a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.341206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.342909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.343194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.343541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.355449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:00.361001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:00.362028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:00.363072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 394/776] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:00.371289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.394532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb4ff2-67c3-446a-85a6-52e2e3990214 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.424361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.433936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.434243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.437761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.437761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.437761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.437761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.437761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.450511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 383/777] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.456979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.518419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.592984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec66b4-c4e9-4981-bebf-ff7e7fa85b49 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.620849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48dad8-2cec-420e-9cf3-e4c5ab9ff768 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.639986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.640596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.640874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.641192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.641896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.642196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.643556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.656685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 395/778] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.656685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.665606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.666522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.667439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.667439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.667944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=fce06803dd124d6680a9ce5792e7ea90) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:00.669569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.669785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.680376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 384/779] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/role_assignments?user.id=fce06803dd124d6680a9ce5792e7ea90&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:00.682017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.770071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.856807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e25d7b-8dbd-4c97-963c-081221ff3f2f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.879848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdf32f-b62a-4453-903d-16d5729d4ab5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:00.915152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:00.925383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.926037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.926320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.926581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.940292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:00.940949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:00.941226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:00.941499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] PATH_INFO: `/v3/services/network` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:00.949840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.949840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.950052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:00.950052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:00.963360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:00.964389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:00.965241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 396/780] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services/network => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:00.967527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:00.969550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:00.971693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:00.971693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:00.971693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:00.975102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 385/781] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:00.975102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:00.978480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.081470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1dcc9-cdd2-4032-a193-1e02ade3c491 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.088981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.116903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.117513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.117845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.118124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.119288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.119591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.120938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.137868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 386/782] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.139692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.249686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e8690-9a09-488d-9dbc-d7aa49aa32f7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.252776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b96bd1-d29a-40f4-a452-4e1ac6681d5e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.286607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.287285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.287569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.287836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] PATH_INFO: `/v3/users/swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.301883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.301883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:01.310650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:01.311592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:38:01.312536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:01.318324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 387/783] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:01.318324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.327344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.328234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.328718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.329192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.330055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] RBAC: Adding query filter params (name=network) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.330561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.332197 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.337568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 397/784] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:01.342618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.358864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.395988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.450616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455b073-c96b-4c78-b6cf-3cc5c3fb25bf None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.503436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.505281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.506473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.507146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.509347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f651639f-f0c6-41c0-8d28-7e10722500bc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.510421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] RBAC: Adding query filter params (type=network) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.510946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.512589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.526865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 398/785] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:01.537801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.594311 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.596571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.597056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.597527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.598338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.598641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.600058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.622897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 388/786] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.628099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.693994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c53b6f-cb0f-4e33-84cc-cb2af2d93640 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.708416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.722434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:01.735656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.736267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.736546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.736817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.737612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] RBAC: Adding query filter params (interface=public, service_id=d215968803a540b1ad070ae208611700, region_id=RegionOne) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:01.738049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.739417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61869e72-ea70-45ad-8875-8bdb1c490318 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.744038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3257a1-ccff-4d03-ac23-8ba1cccee2d1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.745630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 399/787] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/endpoints?service_id=d215968803a540b1ad070ae208611700&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.746866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e50277-f62d-4a71-94f5-4fd9a18d669f None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.747175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e50277-f62d-4a71-94f5-4fd9a18d669f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.747672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e50277-f62d-4a71-94f5-4fd9a18d669f None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.754063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 400/788] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:01.754964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e50277-f62d-4a71-94f5-4fd9a18d669f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.793016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.793695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.794027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.794261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.807157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.808594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-be54f5b9-ac46-4a63-b43b-f8dcd9516f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:01.825609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:01.827874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:01.830059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:01.830059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:01.830059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:01.836545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 389/789] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:01.836545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.836545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.836545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.856954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:01.858659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:01.878236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e50277-f62d-4a71-94f5-4fd9a18d669f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:01.915458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:01.916179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:01.916342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:01.916592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:01.917191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:01.918841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:01.928949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 390/791] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:01.929988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 401/791] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/services => generated 772 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:01.930185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:01.994564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:02.043564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0fd36-1edc-439a-99f4-c06eefa2228b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:02.079850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:02.080467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.080736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.081001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.081652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:02.081982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:02.083338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:02.097060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 391/792] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:02.102738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:02.152812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:02.216593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de8643-965b-438d-b6c5-fa5dee346e48 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:02.251919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:02.252550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.252823 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.253088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.261138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:02.262595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:02.273990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:02.274961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:02.275856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 402/793] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:02.280959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:02.333601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:02.389406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066b2e9-aff5-4aca-8696-5dde010ae84d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:02.426167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:02.444590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:02.445193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.445465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.445747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.446990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:02.447288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:02.448687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:02.478570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 392/794] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:02.484689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:02.499456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df911d-1c66-4340-bfe1-986152b46a39 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:02.501368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:02.569870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:02.570573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.570855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.571193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.590889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:02.596353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:02.597290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:02.598176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 403/795] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:02.605541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:02.678740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fcd913-9998-48f9-855b-343d0d903338 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:02.756367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a0f815-5f5f-4076-9756-dad51b337182 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:02.765499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d90c0-9b26-4f2d-9eab-309511d81452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:02.766209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d90c0-9b26-4f2d-9eab-309511d81452 None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.766605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d90c0-9b26-4f2d-9eab-309511d81452 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.766881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d90c0-9b26-4f2d-9eab-309511d81452 None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/38d08c9aa2604871bec462424708b85b/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.775362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d90c0-9b26-4f2d-9eab-309511d81452 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=38d08c9aa2604871bec462424708b85b, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:02.776813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d90c0-9b26-4f2d-9eab-309511d81452 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:02.793854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:02.794894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.795348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.795808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.796782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:02.797263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:02.799141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:02.803645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 393/796] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:02 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/38d08c9aa2604871bec462424708b85b/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:02.805305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af897f-e85c-4f6f-8667-e05cbddcda04 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.806164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af897f-e85c-4f6f-8667-e05cbddcda04 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.806504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af897f-e85c-4f6f-8667-e05cbddcda04 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.808084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 394/797] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:02.813896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:02.814432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 404/798] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:02.814717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:02.815191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:02.820030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:02.837566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:02.839149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:02.866198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:02.911337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:02.977148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 395/799] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:03.014655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080c916-df17-41d0-a4df-ddbd8851d879 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.070543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.071186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.071466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.071737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.084265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.085810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-dd88c19d-5fd0-412d-9344-21fc4063d5db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:03.103559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:03.104660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:03.105629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:03.105629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:03.105629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:03.105629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 405/800] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:03.112002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.248331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfdfa6-8a66-46fe-947e-9dbcb9d05d12 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.248671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfdfa6-8a66-46fe-947e-9dbcb9d05d12 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.248874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcfdfa6-8a66-46fe-947e-9dbcb9d05d12 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.250367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 406/801] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:03.256539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.256830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.257092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.294430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:03.295645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:03.351903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a594d84-a6dd-4aae-bbbe-0215c4901872 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.388870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 407/802] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:03.390762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.395929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.400433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.401228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.403245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.403676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.404467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:03.405374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.407343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.422197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 396/803] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:03.428049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.472214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.507271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46041c5d-6b39-4560-b7fb-dc53be95d9fd None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.544613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.545252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.545531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.545856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.546442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.547822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:03.557045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:03.558098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:03.558951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 408/804] 10.222.0.15 () {56 vars in 1129 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:03.559764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413c669-abe3-4996-809a-bcc9f0c07ae8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.570819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.608488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.610584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.610860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.611127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] PATH_INFO: `/v3/users/nova` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.628628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.630081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:03.659869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:03.662143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:03.664394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 397/805] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:03.671320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.704126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.739948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842d24e5-e4a2-4ef2-bea6-6a8d65425b45 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.778412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.779033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.779314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.779587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.780297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:03.780599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.781994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.787198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 409/806] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:03.787989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.806940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.816458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aed7cfd-aed9-4e92-b013-11a21f3db9bc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.855987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.856700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.856984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.857259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.858014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:03.858316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:03.859985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:03.880646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 398/807] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:03.881585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.896530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c8a305-2841-4960-a821-d05ea4b76b1c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:03.931414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:03.939177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:03.940113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:03.940402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:03.940677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:03.954998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:03.962298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:03.963214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:38:03.964051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:03.967463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 410/808] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:03.967463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:03.997235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f2c6a-d058-4ddb-98be-04228b44a054 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.035480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.036559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.036842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.037377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.038034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:04.038334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.039944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.047439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 399/809] 10.222.0.15 () {56 vars in 1124 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:04.048246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.094179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.095815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c505b-b4ff-417a-8555-94983135f67b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.141874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.142994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.143645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.143993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.166548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a185bb8d-3b4d-4220-859c-d2624ea57445 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.206634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.207394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.207671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.207984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.208652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:04.208948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.211097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.220698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 400/810] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:04.221475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.231413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.243876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.246221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.317760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.350225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ea755-1d8b-40db-bf1a-a73cbe0ebb3d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.371212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 411/811] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:03 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:04.372155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.395855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.396593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.396873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.397173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.397972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.400185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.432481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 401/812] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:04.433641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.488979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.501548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e4e0a-2c41-4f75-9771-581f19fee998 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.545378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.546067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.546343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.546605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.562003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.563677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.567703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd209a2d-82f9-4c61-9a12-b8009dc307bf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c12a45f3-1010-496c-bd03-fedb02c93cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:04.583621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:04.584668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 412/813] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ce20-b847-49c9-96bb-dc53fac1582b None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ce20-b847-49c9-96bb-dc53fac1582b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4ce20-b847-49c9-96bb-dc53fac1582b None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.589862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 413/814] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:04.594975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.595688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.596203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.610782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.616134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.616784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.617174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.617450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.619638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:04.621940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:04.631104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.632584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:04.648063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:04.649632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:04.650605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:04.650605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:04.650605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:04.654515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 402/815] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:04.655463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.718167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 414/816] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:04.719429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.796454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53925b21-7910-44db-9e49-48ab08a250f2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.811012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.840642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.841160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.841459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.841748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.842398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:04.842701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.844134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.851548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49747dae-62ad-4744-98d2-68ee88911aeb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:04.861879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 403/817] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:04.863641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.890480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:04.891442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:04.891920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:04.892500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:04.893502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:04.894066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:04.895866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:04.909361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 415/818] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:04.912612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:04.915727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:04.973965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb9bcdb-8818-4265-94fa-a86b4b42e527 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.017511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.018153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.018430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.018700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.028201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.029853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.042865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e26b7b-5cc7-49cc-b38b-d8252a39bd35 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:05.044673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:05.045641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:05.046666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 404/819] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:05.047260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.078840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.086167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.087799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.088418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.088992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.097009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.099315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:05.112137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:05.113155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:05.114121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:05.116725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 416/820] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:05.117788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.162933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5bf5df-a950-485d-8aae-e29b8f29eddb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.165100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.201923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.202553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.202828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.203095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] PATH_INFO: `/v3/services/network` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.204181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.205537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:05.216607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:05.217662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 19:38:05.218561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:05.221246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 405/821] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:05.222156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.238619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec4903-2b42-43fa-89e8-f21d4ceff4c9 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.279163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.279821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.280104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.280375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.281106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:05.281406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.283102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.302505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 417/822] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.303410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.329262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2cf114-6df8-461f-b988-644f66960e22 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.358415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.372684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.373291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.373564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.373870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.374557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:05.374852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.376253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.394169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 406/823] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.394169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.429722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31095cbb-bbe8-41a3-ae01-1b0d1e5ad19d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.431757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.479421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.480154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.480499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.481104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.481569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] RBAC: Adding query filter params (name=network) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:05.481968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.483450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.488986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 418/824] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.489851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.523056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84821316-b35e-4c46-a5bd-d1db70305815 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.558971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.559890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.560362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.560826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.572332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.574062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:05.590081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:05.591098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:05.592912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:05.592912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:05.592912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:05.592912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 407/825] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:05.592912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.601303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b65ae-dc51-41ff-90e7-a78c6104531b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.615756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.641280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.642204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.642491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.642763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.643683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=910ca69bc9bf4f328d9537f0b9bb1658) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:05.644282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.645724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0f18d3-0c95-48c2-b9e2-3ec7ebf47d8d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.653953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 419/826] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/role_assignments?user.id=910ca69bc9bf4f328d9537f0b9bb1658&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.655996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbc7ee-07ef-4c3a-ab3f-c91c22dc20a1 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.656287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbc7ee-07ef-4c3a-ab3f-c91c22dc20a1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.656813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffbc7ee-07ef-4c3a-ab3f-c91c22dc20a1 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.665817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 420/827] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:05.666166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbc7ee-07ef-4c3a-ab3f-c91c22dc20a1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.684377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.708248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e5de4-0638-441d-bc33-4340d0a525c6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.750280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.750900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.751282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.751563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.752276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] RBAC: Adding query filter params (type=network) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:05.752675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.754140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3c81d-54be-47c3-ad1d-023877b9f4bb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.761936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 408/828] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.763654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.764041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.764414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.785884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:05.787258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:05.813300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffbc7ee-07ef-4c3a-ab3f-c91c22dc20a1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:05.860863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:05.861652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:05.861782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:05.862041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:05.862673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:05.862970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:05.864330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:05.877450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 409/829] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:05.877952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 421/830] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:05.878034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.887179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:05.933702 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.966153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:05.994891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784d33a-2461-432e-b730-4d5f5f9e6147 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.027518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e1101e-69d9-4931-b35b-a0a774929e6b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.039390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.040491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.040763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.041033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.041701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.043483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.078749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 410/831] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:06.091067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.092167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.092698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.093172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.101669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.104626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:06.118172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:06.119174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:06.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:06.122634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 422/832] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:06.128871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.165513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.296851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.318988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8046dc5-7d0b-4f8a-8aef-3ba87b290285 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.324590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b230-fa2b-4b04-98df-8a53f940edf8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.366978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.372019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.373000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.373484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.374778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.375460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:06.376540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.378354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.382835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.384250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.384680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.385108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.393849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 411/833] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:06.399465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:06.400209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:06.406331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:06.407519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:06.408569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:06.414483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 423/834] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:06.417045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.521881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a5f9d2-aa15-47c5-b318-b8c91f6119bf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.553003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b35a12-1ccc-4a58-861a-75ec3befd59c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.566255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.566826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.567168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.567553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/fce06803dd124d6680a9ce5792e7ea90/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.584965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=fce06803dd124d6680a9ce5792e7ea90, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.587137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.598003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.599705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.600192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.600916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.601765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:06.602322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.603878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.604267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 412/835] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:06 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/fce06803dd124d6680a9ce5792e7ea90/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:06.618987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 424/836] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:06.621682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.677286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.746873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ee47c-298a-4ce0-8ede-abb0d647267b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.786068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.787038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.787500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.787953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.799352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.801069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:06.817406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:06.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:06.819510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:06.819510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:06.819510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:06.821760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 413/837] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:06.826764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:06.847286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:06.945067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569443b-7c99-4a37-af6c-600280afe2cc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:06.984467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:06.985494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:06.986010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:06.986480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:06.987337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:06.987839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:06.989475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420ddebb-3efb-4f71-b69b-bb7e89123c65 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.003722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 425/838] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:07.009760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.096159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981708-73ad-40fa-bc32-804c3672033e None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.096603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981708-73ad-40fa-bc32-804c3672033e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.096983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c981708-73ad-40fa-bc32-804c3672033e None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.099249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 426/839] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:07.107417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.107989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.107989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.112244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.122558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a6297-c5ea-4fa2-9f40-52f2917d24b1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.129383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:07.131257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:07.159441 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.160066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.160456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.160738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] PATH_INFO: `/v3/users/swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.168956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.170635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:07.182708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:07.183706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 19:38:07.184741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:07.188647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 414/840] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:07.189848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.218172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 427/841] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:07 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:07.302002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae33577-14b1-487c-a960-92602a7425aa None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.342648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.343254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.343594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.343909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.344651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.344954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.346409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.369885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 415/842] 10.222.0.15 () {56 vars in 1135 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:07.374828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.378369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.479848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15cf30-556f-4e96-a3e6-ae3fe710503b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.515256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.515432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.515979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.516275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.516650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.529316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.615888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.618188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.621547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.637480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b17b5-d915-4a0f-ada8-34bc81e90544 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.676606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.677693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.678193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.678909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.679679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.681472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:07.691084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:07.692088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:07.692934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:07.700213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 416/843] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:07.701786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.796505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:07.828383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b45dfa-4db4-4e86-a409-6ebbf437a0ab None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:07.866163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:07.867048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:07.867363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:07.867638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:07.868268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:07.868574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:07.870442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:07.880680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 417/844] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:07.881711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.883378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 428/845] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:07.884642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:07.995642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5cd14a-1891-4384-bad1-cf2841805110 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.018498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f766232-1c2d-454f-88a4-a985c0825593 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.021393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.036646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.037968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.038473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.039050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.040033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.040701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.042289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.054185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 418/846] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.058967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.059637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.059918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.060192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.074282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.075809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-58623ab0-3a76-47c3-83b1-7274ff6be889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:08.093117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:08.094186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:08.095078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:08.095078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:08.095078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:08.099099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 429/847] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:08.100530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.104301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.187395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a6f5f-d383-4242-ab5a-7dce23ebb36a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.187928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a6f5f-d383-4242-ab5a-7dce23ebb36a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.187928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a6f5f-d383-4242-ab5a-7dce23ebb36a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.189552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 430/848] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:08.195255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.196082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.196360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.216277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:08.217477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:08.222783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840ea0a-b19b-45fd-984f-7872272922d8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.280335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.280948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.281221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.281486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.282158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.282451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.283799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.296308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 419/849] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.299572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 431/850] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:08.302441 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.368200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.411906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f3d49e-589f-418c-bb56-96a918d30387 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.452229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.452856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.453132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.453401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.460381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.462012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:08.473429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:08.474391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:08.475557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 420/851] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:08.482546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.577721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.596123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd0d4d3-dae8-4628-8fda-e5144af753ed None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.614652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.636294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.636829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.637106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.637380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.638252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.638555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.640268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.659764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 432/852] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.662105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.733622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb07d45-88e2-43b9-9385-6a14f475cda5 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.761044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.781581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.781581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.781581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.781581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.782169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.782169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.789778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36456216-2fa0-451c-a55f-8212900a2fa7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:08.791877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:08.792829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:08.794132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 421/853] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:08.800958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:08.831131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.831775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.832054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.832328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.833215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=38d08c9aa2604871bec462424708b85b) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.833529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.836149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ed92aa-b705-4f08-a584-ff0aff9ddb0a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.853048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 433/854] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/role_assignments?user.id=38d08c9aa2604871bec462424708b85b&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:08.864720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612823ca-7751-4226-bc61-d2e6dd71bd60 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.865011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612823ca-7751-4226-bc61-d2e6dd71bd60 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.865283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612823ca-7751-4226-bc61-d2e6dd71bd60 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.867749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 434/855] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:08.873603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.874222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.874769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.894984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:08.896459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:08.924203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:08.935904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ffb694-7fd4-4d51-a928-4580e7788628 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:08.978156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 435/856] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:08.984501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:08.985186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:08.985463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:08.985820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:08.986478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:08.986784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:08.988246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:08.994826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 422/857] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:09.000634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.108488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f7923-2ee9-43b7-89c9-abeb4418e8c4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.143155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.144150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.144656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.145182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.146107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:09.146642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:09.148195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:09.157889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 436/858] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:08 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:09.246796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.301804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.355945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f2940f-5549-424d-b0ac-fa0daf17e157 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.391808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.392437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.392708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.392970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.404268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:09.408878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:09.409899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:09.410774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 423/859] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:09.416177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.420192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.527055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a37e95-81dd-4106-92ec-cec8902933f7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.562199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.562860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.563144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.563418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.564082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:09.564383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:09.565753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:09.574920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 437/860] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:09.580412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.682452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31d2d16-183e-4169-8975-d1b11364dfb2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.698699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.728514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.729529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.730038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.730516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.741445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:09.743716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:09.758718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:09.759678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:09.760564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:09.760564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:09.760564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:09.762596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 424/861] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:09.768378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:09.845526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:09.885357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ce98e-db05-43e8-a43d-cddf3bad93e1 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:09.922360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:09.922967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:09.923250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:09.923502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:09.924129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:09.924427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:09.925792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:09.939216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 438/862] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:09.944279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.050047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705c270-9fb1-4a7e-ba87-fd0017d62f57 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.085023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.085640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.085968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.086240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.093744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.095172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:10.106088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:10.107010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:10.107980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 425/863] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:38:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:10.112820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.227209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0b550e-6ea5-4747-aa92-d96fc5b693ec None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.256001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.265836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.266444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.266716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.266985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.267697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:10.267995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.269405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03286241-c29f-4d7e-b80f-60906e60e65b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.287064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 439/864] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:10.293152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.321571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.340382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de80cc-2f27-41ef-a5b9-28b23291b729 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.340668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de80cc-2f27-41ef-a5b9-28b23291b729 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.340945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8de80cc-2f27-41ef-a5b9-28b23291b729 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.342438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 440/865] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:10.348780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.349358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.349636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.370576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:10.372076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:10.406309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872bcbf8-1c86-4298-8a8a-f2e83b36de6c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.447282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.447883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.448160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.448430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.449866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 441/866] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:10.526444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.528258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.552393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.636011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 426/867] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:10.641817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.744989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.749539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8fc57-f26b-4640-b828-b8a5642f7a34 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.766185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.792159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.793125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.793409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.793851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.807955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.809940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:10.830300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:10.831425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:10.832409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:10.832409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:10.832409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:10.832409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 442/868] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:10.837567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:10.864468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fda06d9-d944-4309-a410-c5aacbe84cc0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:10.890577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:10.904587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:10.905588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:10.906109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:10.906591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:10.907452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:10.909200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:10.937766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 427/869] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:10 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:10.970366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab163ce1-4be5-40a2-af6a-28a263c48c7e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.022044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.023079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.023622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.024095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.025017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:11.025530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.027275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6250ac7-a6d7-4da7-a930-fd23bc6d46eb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.042742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 443/870] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:11.054154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.175373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cc768b-2f0c-49e9-8440-7ca4372ee65b None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.211780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.212362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.212643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.212914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.213628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dd132-bccb-40e5-a9fa-fc7efcc6739a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.213943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dd132-bccb-40e5-a9fa-fc7efcc6739a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.214224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77dd132-bccb-40e5-a9fa-fc7efcc6739a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.215737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 444/871] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:11.220987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.221512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.222038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.223503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.224936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:11.237962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:11.238899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:11.239799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 428/872] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:11.245083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:11.246326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:11.246631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.264514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.298850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.325657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 445/873] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:11.367297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a52d646-28ad-4b71-8736-25974f1a819f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.403277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.404251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.404742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.405320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.406247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:11.406811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.408435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b3f25-684a-4c7a-bd21-3b763bfa1797 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.422866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 429/874] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:11.429937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.542229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3874fa98-6fd9-4a54-afb3-85aa3775c339 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.545191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f7b4b-7eee-4380-986a-5b3efca7fec4 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.545472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f7b4b-7eee-4380-986a-5b3efca7fec4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.545756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f7b4b-7eee-4380-986a-5b3efca7fec4 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.547344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 430/875] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:11.552313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.552530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.552774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.571248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:11.572419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:11.582091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.583107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.583598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.584085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.585023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=fce06803dd124d6680a9ce5792e7ea90) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:11.586154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.587914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.595949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 446/876] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/role_assignments?user.id=fce06803dd124d6680a9ce5792e7ea90&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:11.621643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.646195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.662032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 431/877] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:11 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:11.739639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4731dd9-0a1e-4d84-9749-98f89f2c060e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.778291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.778937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.779216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.779484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.780072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.781439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:11.790987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:11.792085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:11.793371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 447/878] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:11.798414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.841522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:11.915452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f044e3-343b-427e-abaf-7bfd122f8d2a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:11.937877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:11.956179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:11.956785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:11.957055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:11.957320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:11.957976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:11.958271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:11.959621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:11.966808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 432/879] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:11.972096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:12.033466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.053786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f43823-52cc-4421-8271-a9b4c3583217 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.084279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:12.089997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856a093-2c7c-40ff-9f27-f3f10bf254ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.090543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856a093-2c7c-40ff-9f27-f3f10bf254ef None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.090819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856a093-2c7c-40ff-9f27-f3f10bf254ef None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.091090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856a093-2c7c-40ff-9f27-f3f10bf254ef None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.091751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856a093-2c7c-40ff-9f27-f3f10bf254ef None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.092525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc23bcc-2715-496d-a79c-3e749713c55d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:12.093105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856a093-2c7c-40ff-9f27-f3f10bf254ef None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.117760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 448/880] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:11 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:12.141837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:12.142495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:12.142820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:12.143095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:12.143742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:12.144102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:12.145553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4436271-7e32-4708-82e4-016a394a73ad None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:12.151819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 433/881] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:11 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:13.208939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99575b-6360-408f-bee9-d0023dc4027a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.209433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99575b-6360-408f-bee9-d0023dc4027a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.209433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99575b-6360-408f-bee9-d0023dc4027a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.211467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 449/882] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:13.216875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.217295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.217462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.235677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:13.236859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:13.306600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 434/883] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:13 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:13.580667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99575b-6360-408f-bee9-d0023dc4027a None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.690055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99575b-6360-408f-bee9-d0023dc4027a None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.725327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.726038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.726318 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.726585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.727164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.728528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:13.737708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:13.738768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:13.740172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 450/884] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:13.747649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.855292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:13.874267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dcf44b-1811-457b-a10c-b3ab9df0ba0f None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:13.913016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:13.914212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:13.914524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:13.914757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:13.915391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:13.915687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:13.917044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394cecd-542b-41f6-aaa0-6ca7e47fe414 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:13.922714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 435/885] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:13.930833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:13.939116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.000046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9758de-d693-4e3c-b891-d8b3e24cd11c None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.000396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9758de-d693-4e3c-b891-d8b3e24cd11c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.000579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9758de-d693-4e3c-b891-d8b3e24cd11c None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.002457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 436/886] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:14.009007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.009557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.010061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.029478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:14.031039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:14.054657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6620a-433f-4e24-840a-d47ab375362f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.101818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.102335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.102612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.102882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.103510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:14.103811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.109753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.110256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 437/887] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:14.111226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 451/888] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:13 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:14.115997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.201218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.221556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d14cce-de4e-4b53-8667-64da3d283fb4 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.269792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.269792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.269792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.269792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.270495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] RBAC: Adding query filter params (interface=public, service_id=c843e5dceb57488f8fd30c74ef2774af, region_id=RegionOne) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:14.270495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.270495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdaadc8-f989-4849-8795-901584a6b163 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.280575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 438/889] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/endpoints?service_id=c843e5dceb57488f8fd30c74ef2774af&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:14.286974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.371615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.396627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0924-26e0-4191-b7e0-c25219ba925f None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.396912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0924-26e0-4191-b7e0-c25219ba925f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.397188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e0924-26e0-4191-b7e0-c25219ba925f None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.399623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 439/890] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:14.405832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14216ccc-ebdc-47b8-8866-dc98300bb2a3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.406116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14216ccc-ebdc-47b8-8866-dc98300bb2a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.406393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14216ccc-ebdc-47b8-8866-dc98300bb2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.409639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953a048b-019d-491c-8ba8-0b8e2be5d06b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.427737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-14216ccc-ebdc-47b8-8866-dc98300bb2a3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:14.429220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-14216ccc-ebdc-47b8-8866-dc98300bb2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:14.448424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.449029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.449307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.449575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.450206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.451556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.457030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 452/891] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:38:14.457813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.503074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 440/892] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:14.504725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99997-58ec-4927-8129-4dbae6a7eb3c None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.505010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99997-58ec-4927-8129-4dbae6a7eb3c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.505299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f99997-58ec-4927-8129-4dbae6a7eb3c None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.511021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 441/893] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:14.512422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.512704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.512975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.533631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:14.534885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:14.606775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f998ef5-e064-4832-b111-5f436d2fb37e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.644391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.645694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 442/894] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:14 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:14.653448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.654167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.654450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.654721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.665257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:14.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:14.671020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:14.671862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 453/895] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:14.679307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.683218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.785481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.785924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac0da01-8952-48cd-8614-156b4901d257 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.823274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.823876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.824152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.824423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.825087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:14.825838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.827204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.840910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 443/896] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:14.846605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:14.927251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2489c42e-7b9f-476f-9bee-04de35d78cad None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.929101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:14.968237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292ad8c-317e-402e-9101-d193215c3e07 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:14.971369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:14.971978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:14.972252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:14.972518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:14.973169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:14.974554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:14.990303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 454/897] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:14 2022] POST /identity/v3/services => generated 241 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:14.992123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.011705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.012661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.013095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.013534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.026400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.028113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:15.047359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:15.048436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:15.049345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:15.049345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:15.049345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:15.049345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 444/898] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:15.055294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.081667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.154781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98f4cc-b6bd-4ce0-8d74-12e5f22f6d2a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.183260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b614-fa95-4a3f-89be-852749048dcf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.202777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.207978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.208672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.209134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.209546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.210316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.211841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:15.220923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:15.222003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:15.223072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 455/899] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Sep 13 19:38:14 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:15.228697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.235873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.237168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.237447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.237713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.238378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:15.238957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.240055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.253275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 445/900] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:15.259413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.344248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913cdd2e-3c2f-4587-87ff-9c8d2c1a54f5 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.359939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9989f-d7bc-46c3-8b6c-ca04d5c15234 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.363780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.386302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.387215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.387648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.388117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.389009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:15.389470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.391106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.396183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.396898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.397069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.397216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 456/901] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:15.397317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.404470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.405787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.406542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:15.419834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:15.420881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:15.422101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:15.426350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 446/902] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:15.427327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.467168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.520604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04851472-2354-43c9-8613-4b333231fb8b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.535033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22947e3-109c-44aa-8e5c-72affe700b2a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.558464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.559103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.559383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.559656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.560280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:15.560582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.562152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.567665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 457/903] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:15.573152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.579614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.580224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.580502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.580774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.581503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:15.581847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.583266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.605778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 447/904] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:15.610455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.666129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.687979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd029c1-fb30-458b-b792-bd3c248d083d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.724148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.734398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.734555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cc607f-e373-44c1-b8f8-aefac1c99bde None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.735581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.736415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.737291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.738208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] RBAC: Adding query filter params (interface=public, service_id=a1d4c774bc8a4686a317b951bdb4b789, region_id=RegionOne) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:15.738701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.740273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.747121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 458/905] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/endpoints?service_id=a1d4c774bc8a4686a317b951bdb4b789&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:15.752194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:15.776977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.777698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.778087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.778429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.860036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.861711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.868856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d518ce9e-a3e5-434d-b233-0a30970e1812 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:15.879877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.909787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:15.910744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:15.911203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:15.911687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:15.912577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:15.914802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:15.921716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 459/906] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:38:15.974403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:15.987592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 448/907] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:15.997091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.108092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb3158-7461-4106-8123-68e871597bac None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.146012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.146942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.147404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.148499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.159370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.161186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:16.176325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:16.177343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:16.178533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:16.178533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:16.178533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:16.180258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 460/908] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:16.185766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.289807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb89030-22db-4f2d-954a-4578a5ee61a7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.325291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.325922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.326192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.326453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.327091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.327382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.328799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.341614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 449/909] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:16.346754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.373274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.451394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ae583-3772-4fd6-8781-3fc404fd599e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.489793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.490477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.490758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.491029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.498448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.499936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:16.512067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:16.512981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:16.514187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 461/910] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:16.519283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.575603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.624490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd5f9b-9774-4ac6-8811-991cb9908580 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.659262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.659892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.660166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.660434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.661143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.661442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.662934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e166e65-d4ad-40de-86d5-9c53efa1d0f8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.677492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 450/911] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:16.684557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:16.773919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:16.786771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5d97b-1167-4f2d-a08d-6432d10404c0 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.787088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5d97b-1167-4f2d-a08d-6432d10404c0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.787327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c5d97b-1167-4f2d-a08d-6432d10404c0 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.789108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 451/912] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:16.793917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5f9df-f627-4c3a-a700-88dc01466a06 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.794197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5f9df-f627-4c3a-a700-88dc01466a06 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.794469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc5f9df-f627-4c3a-a700-88dc01466a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.799595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97616b9-a5fd-4ce8-9518-fcdb1d10ddbc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:16.813799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6dc5f9df-f627-4c3a-a700-88dc01466a06 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:16.814991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6dc5f9df-f627-4c3a-a700-88dc01466a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:16.846553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:16.847709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:16.848134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:16.848708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:16.849927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] RBAC: Adding query filter params (role.id=bf95583bb5cc4f7b80ea496902da57f4, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=fce06803dd124d6680a9ce5792e7ea90) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:16.850518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:16.852388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:16.860196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 462/913] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:16 2022] GET /identity/v3/role_assignments?user.id=fce06803dd124d6680a9ce5792e7ea90&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=bf95583bb5cc4f7b80ea496902da57f4 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:16.904775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 452/914] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:16 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:17.187255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.294725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3e423d-6533-42c6-843d-da28faa68e0d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.299187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93859f88-58d1-4b87-a40a-d6f17a2aed9a None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.299687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93859f88-58d1-4b87-a40a-d6f17a2aed9a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.299687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93859f88-58d1-4b87-a40a-d6f17a2aed9a None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.301551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 453/915] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:17.309155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.309477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.309693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.330232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:17.331443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:17.347083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.348678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.349158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.349888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.351829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.353417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:17.366489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:17.367551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 19:38:17.368404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:17.371642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 463/916] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:17.377062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.384867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.403876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.428214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 454/917] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:17 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:17.496406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096a4b5-12c8-452d-9e7c-7a14d38dd13d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.530799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.531648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.532068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.532476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.533229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:17.533667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.535195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.540988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 464/918] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:17.546321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.656025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1549e-69f7-4e7d-901a-fc9f39ed4b06 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.691668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.692276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.692555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.692825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.693451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:17.693769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.695767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.701234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 455/919] 10.222.0.15 () {56 vars in 1114 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:17.706495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.709685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.750276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.807964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:17.829469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004ace2-3a81-4248-90d0-6a9582625764 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.834367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf719f42-9cf7-4613-8f36-5946e9e5b755 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:17.872210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.872521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:17.873007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.873354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.873605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:17.873764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.874218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:17.874493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.874790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:17.875584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:17.876050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.877226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:17.885970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:17.887170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:17.888138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:17.893539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 456/920] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:17.895192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:17.905325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 465/921] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:17 2022] POST /identity/v3/endpoints => generated 346 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:18.018936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:18.022404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622b6683-928e-4537-9074-a87a24de9df7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:18.067643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:18.068256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.068535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.068801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.070142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:18.070446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:18.071900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:18.081883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 457/922] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:17 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:18.083408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:18.196090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a034e-3f55-4536-8612-b3d2630a830a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:18.226717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:18.232135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:18.232742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.233023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.233363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.234048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:18.234557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:18.236217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb04d5b-2714-4869-89ab-5b780ff21c52 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:18.242659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 466/923] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:18.248965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:18.251131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e510-04f6-440b-b2ef-129a4b0e7f7f None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.251648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e510-04f6-440b-b2ef-129a4b0e7f7f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.252137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6e510-04f6-440b-b2ef-129a4b0e7f7f None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.253900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 467/924] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:18.261930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.262455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.262948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.283127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:18.284347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:18.359340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927a8a8c-cf63-472f-b7ba-2467b4ecd918 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:18.373767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 468/925] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:18 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:18.399549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:18.400197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.400634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.401054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.402012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] RBAC: Adding query filter params (interface=public, service_id=9cfff7ff45a9483184584e566736811d, region_id=RegionOne) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:18.402471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:18.403959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:18.409909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 458/926] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:18 2022] GET /identity/v3/endpoints?service_id=9cfff7ff45a9483184584e566736811d&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:18.416031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:18.524228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bef88a-08c0-4b98-92af-da07c3189bb8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:18.551063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:18.561463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:18.562101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.562380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.562654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.563260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:18.564615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:18.570954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 469/927] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:18 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:38:18.667676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:18.697588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:18.791826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddbe984-f97e-4a2f-af2b-85abefd19281 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:18.829228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:18.829807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:18.830090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:18.830362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:18.830951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:18.836996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:18.851308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:18.852518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 19:38:18.853634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:18.855966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 459/928] 10.222.0.15 () {56 vars in 1132 bytes} [Tue Sep 13 19:38:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:18.861790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:18.974283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93d8a6-c241-4a08-958d-9d5dfbc7ff78 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:19.011041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:19.011884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.012312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.012733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.013498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:19.014000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:19.015502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ea2b1-89e3-4f8a-872b-66899c108508 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:19.020817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 470/929] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:19.030929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:19.129169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:19.141455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db289dad-9e9c-4c90-a4b2-54917d2e2a28 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:19.153097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935cf9f-a559-42d8-beb9-7653a9b144b5 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.153380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935cf9f-a559-42d8-beb9-7653a9b144b5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.153653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935cf9f-a559-42d8-beb9-7653a9b144b5 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.155348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 471/930] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:19.160182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.160459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.160726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.175864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:19.176463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.176738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.177013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.177911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:19.178213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:19.179602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:19.182642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:19.184255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:19.185176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 460/931] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:19 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:19.190299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:19.246841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:19.276898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 472/932] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:19.308723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f00667-1039-4795-9438-d1a2c170607e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:19.344581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:19.345503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.346003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.346449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.347317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:19.348842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:19.373009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 461/933] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:19 2022] POST /identity/v3/endpoints => generated 347 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:19.575620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:19.677483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45004508-0fdd-4a8f-8f80-8b4964e10d0d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:19.718939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:19.719977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.720456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.720952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.731485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:19.736514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:19.737434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:19.738356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:19.741428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 473/934] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:19.747116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:19.759503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:19.858582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d6a2a-e1f8-4f9b-932f-b0e9929f8013 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:19.894244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:19.895292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:19.895551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:19.895816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:19.896479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:19.896771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:19.898193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:19.906897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 462/935] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:19.912449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:19.966099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.018254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c55e4e2-71ba-4c72-9739-1ba4bed72c13 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.053167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.054438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.054714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.054982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.064989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.066452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:20.080793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:20.081842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:20.082761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:20.082761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:20.082761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:20.082761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 474/936] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:20.087844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.157724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.193895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa696bd3-edd2-422c-bd43-08000dbbd2d7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.229478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.230644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.230920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.231183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.231812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:20.232110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.233446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052e70b7-edd8-4cdd-8897-f9fc695e6bde None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.246817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 463/937] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:20.252410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.297965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e3a15-7d63-4cd2-a629-f422113d1f58 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.298278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e3a15-7d63-4cd2-a629-f422113d1f58 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.298839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e3a15-7d63-4cd2-a629-f422113d1f58 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.300681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 464/938] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:20.306560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.306833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.307402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.331491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:20.332715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:20.354860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.365176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c374a8-f6f2-497b-8cdb-b579abc6b1a7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.403755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.404930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.405354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.405787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.416786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.418422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.425885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 465/939] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:20.430860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:20.431999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:20.433157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:20.435228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 475/940] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:20.440942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.514164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.545014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5b476-9ef3-4862-91d5-fe1e6207daf2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.579593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.580438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.580855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.581264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.582164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:20.582606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.584148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.602714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 466/941] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:20.608472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.696167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.717597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b102b85-f325-4875-b317-e0dfb454eb5b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.760658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.762086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.762532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.762959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.778328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.780141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:20.796026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:20.797253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:20.798406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:20.798406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:20.798406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:20.803935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 476/942] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:20.805216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.810746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.821461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc9e653-01b5-464f-ac19-29f04de90e83 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.849974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:20.870185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.871175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.871635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.872086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.873005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.874652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:20.877645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING keystone.server.flask.common [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 19:38:20.898711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.sql.core [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:38:20.898711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:38:20.898711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [parameters: {'id': '1b88a56de2e044e9a48d7fd59dd0e371', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:38:20.898711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93113) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-5d3dd52f-25fa-4ed6-946e-da17f4f6b1b1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b88a56de2e044e9a48d7fd59dd0e371.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b88a56de2e044e9a48d7fd59dd0e371. Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:38:20.917666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result.read() Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 19:38:20.918682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application yield resource Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise self.value Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application session.commit() Sep 13 19:38:20.919836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:38:20.920856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:38:20.922071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result.read() Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:38:20.922985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [parameters: {'id': '1b88a56de2e044e9a48d7fd59dd0e371', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.923973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:20.924970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1b88a56de2e044e9a48d7fd59dd0e371. Sep 13 19:38:20.926144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:20.926731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70f691-8eb0-42fe-a6fc-44a0795dfbb7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:20.928719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 467/943] 10.222.0.15 () {60 vars in 1140 bytes} [Tue Sep 13 19:38:20 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:20.930520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560ffd6-a74b-4b53-9383-72d607f82104 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.930972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560ffd6-a74b-4b53-9383-72d607f82104 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.931425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7560ffd6-a74b-4b53-9383-72d607f82104 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.936478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 468/944] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:20.937467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560ffd6-a74b-4b53-9383-72d607f82104 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:20.985972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:20.986885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:20.987378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:20.987841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:20.988992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:20.989519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:20.992814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a86ce5f-62f6-4cba-a462-677a9ded87e9 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.008418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 477/945] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:21.009586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.009940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.010220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.028476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:21.029682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:21.083579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7560ffd6-a74b-4b53-9383-72d607f82104 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.106035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:21.115206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 478/946] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:21.119036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.122024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.122783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.124074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.124074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.132523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.134336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:21.146791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:21.147908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:21.149038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:21.155258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 469/947] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:21.156219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.196383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:21.255162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a24bc8-fc68-4fa0-a4a8-0265b178634d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.276587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add465c0-1c64-4d3f-967f-9193b29b4609 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.309876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.311040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.311859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.312290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.316553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.317252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.317639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.318027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.318773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:21.319703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.321227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.322107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.324005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:21.338433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:21.339557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:21.340680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:21.342719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 479/948] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:21.345894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 470/949] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:21 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:21.351661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.387148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.431758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:21.501765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde92e2-e360-41b3-81b8-6115873878e2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.504039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:21.518155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a108883-9793-458e-9ab5-244dbc9276d0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.548470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.549911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.550196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.550470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.551199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:21.551507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.555428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.571484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 480/950] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:21.578113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.578394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.579714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.579714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.580044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.580744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.582163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:21.592113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:21.593205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:21.594082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 471/951] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:21.599709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.703306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873ad2c-c709-4f8f-a23d-c9c1aa39078f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.719634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:21.724440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe39b8-9ac1-43e9-b483-71154084d42f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.745017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60c4f2c-ad4d-47f2-b2db-b403ae678eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.746125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c4f2c-ad4d-47f2-b2db-b403ae678eaf None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.746405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c4f2c-ad4d-47f2-b2db-b403ae678eaf None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.746798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60c4f2c-ad4d-47f2-b2db-b403ae678eaf None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/fce06803dd124d6680a9ce5792e7ea90/roles/bf95583bb5cc4f7b80ea496902da57f4` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.749573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c4f2c-ad4d-47f2-b2db-b403ae678eaf None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=fce06803dd124d6680a9ce5792e7ea90, role_id=bf95583bb5cc4f7b80ea496902da57f4)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.751032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60c4f2c-ad4d-47f2-b2db-b403ae678eaf None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.760352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.760914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.761191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.761546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.762100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:21.762401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.763741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bccd21-2333-43cb-981f-2bddc1653888 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.773753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 481/952] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:21 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/fce06803dd124d6680a9ce5792e7ea90/roles/bf95583bb5cc4f7b80ea496902da57f4 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:21.773753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466cb3d-57a5-418f-8f01-8fa9ed992420 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.773753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466cb3d-57a5-418f-8f01-8fa9ed992420 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.773753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466cb3d-57a5-418f-8f01-8fa9ed992420 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.774140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 482/953] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:21.774580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 472/954] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:21.775010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466cb3d-57a5-418f-8f01-8fa9ed992420 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:21.775877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.776106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.776379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.797243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:21.798890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:21.821716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:21.882199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 473/955] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:21.913332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466cb3d-57a5-418f-8f01-8fa9ed992420 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:21.952605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:21.953272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:21.953552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:21.953848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:21.954481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:21.954778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:21.956165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:21.962671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 483/956] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:21.968903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:22.076139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c12087-a1e4-4d7d-90eb-8cbed0c01c12 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:22.110774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:22.111369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:22.111636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:22.111954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:22.112616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:22.114044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:22.138271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 474/957] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:21 2022] POST /identity/v3/endpoints => generated 349 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:22.152370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:22.204650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:22.278500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d9a79b-9fde-48d1-9f69-54ef017889dc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:22.322371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:22.323073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:22.323348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:22.323620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:22.324235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:22.325708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:22.336048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:22.337130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:22.338083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 484/958] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:22 2022] GET /identity/v3/services/compute => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:22.343096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:22.356315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:22.463641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5992097e-4a05-43da-b003-cc93a9b3d140 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:22.499379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:22.500212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:22.500635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:22.501075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:22.501930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:22.502368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:22.503849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b60739b-9450-48df-9761-82d5d3a635d4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:22.509838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 475/959] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:22 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:22.516054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:22.616162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:22.622409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899eea3-9c0e-4c41-88ce-056c8b6cc836 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:22.661566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:22.662354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:22.662763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:22.663148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:22.663868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:22.664264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:22.665708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673d355-c311-43fb-ac61-d6af08326d42 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:22.670809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 485/960] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:22 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:23.689151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee43a3-b642-4ebe-a8ab-100319b59650 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:23.689670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee43a3-b642-4ebe-a8ab-100319b59650 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:23.689670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ee43a3-b642-4ebe-a8ab-100319b59650 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:23.691153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 476/961] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:23.697057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873fbd-2d2a-4be9-92cd-63726ce2ff18 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:23.697885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873fbd-2d2a-4be9-92cd-63726ce2ff18 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:23.698374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c873fbd-2d2a-4be9-92cd-63726ce2ff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:23.716908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4c873fbd-2d2a-4be9-92cd-63726ce2ff18 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:23.718385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4c873fbd-2d2a-4be9-92cd-63726ce2ff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:23.789402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 486/962] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:24.051094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80803dbc-a208-4ae6-93e3-d9d86b51152b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.051442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80803dbc-a208-4ae6-93e3-d9d86b51152b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.051667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80803dbc-a208-4ae6-93e3-d9d86b51152b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.054080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 477/963] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:24.059346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.059633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.059902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.065993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80803dbc-a208-4ae6-93e3-d9d86b51152b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.078849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:24.080021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:24.148649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.150475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 487/964] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:24.181854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80803dbc-a208-4ae6-93e3-d9d86b51152b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.215870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.216810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.217281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.217764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.218589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:24.220187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:24.235696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.sql.core [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:38:24.235696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:38:24.235696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [parameters: {'user_id': 'f5cf71123f544567a129859cf762ee64', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:38:24.235696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=93113) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-28965844-3627-415d-aa2c-9e55a5dd1259 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result.read() Sep 13 19:38:24.244169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:38:24.245026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application yield resource Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application next(self.gen) Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise self.value Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application session.commit() Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 19:38:24.246042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.session.flush() Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application compat.raise_( Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 19:38:24.246931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 19:38:24.247835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application conn.query(q) Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application result.read() Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 19:38:24.250464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f5cf71123f544567a129859cf762ee64', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:24.252538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 19:38:24.254409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 19:38:24.257091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 19:38:24.257091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 19:38:24.257091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 19:38:24.257091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 19:38:24.257091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 19:38:24.257091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:24.259136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 478/965] 10.222.0.15 () {60 vars in 1131 bytes} [Tue Sep 13 19:38:24 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:24.267779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.299297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.382654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6ef310-3cb3-4173-9a41-622206970e24 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.414136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefb715-30e9-460f-bdc6-e1525579da7b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.414438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefb715-30e9-460f-bdc6-e1525579da7b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.414737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbefb715-30e9-460f-bdc6-e1525579da7b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.420655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.421359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.421697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.422055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.423184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 479/966] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:24.424146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefb715-30e9-460f-bdc6-e1525579da7b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.430734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:24.432159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-172e2900-0c5b-4baa-ab45-f4b102513a88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:24.443657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:24.444605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:24.445630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 488/967] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:24.446293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.446293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.446455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.464011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:24.465182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:24.538107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 489/968] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:24.539021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.540007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbefb715-30e9-460f-bdc6-e1525579da7b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.554113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.583502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.584170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.584441 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.585156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.587642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.600575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:24.606149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:24.607078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:24.607925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:24.608589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 480/969] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:24.614174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.657117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa90c5a-fdd3-4def-b6b4-a832015dbfc1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.692498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.693097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.693403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.693676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.694400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:24.694699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:24.696107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:24.730841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 490/970] 10.222.0.15 () {56 vars in 1101 bytes} [Tue Sep 13 19:38:24 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:24.736298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912eda91-dace-4341-bbe8-d9c3b758d8b4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.776857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.777472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.777757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.778042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.778780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:24.779073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:24.780760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:24.791921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 481/971] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:24.798960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.863821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:24.886066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.934625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:24.936134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37393fa-a128-4cef-bb69-b69832a84355 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.975716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:24.975716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:24.975987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:24.976232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:24.985081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076f758-ac51-4863-b731-4a90c5338ab3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:24.989425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:24.990677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-9a910f17-632f-41e1-bc12-2fc915b520b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:25.007004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:25.008019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:25.008903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:25.008903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:25.008903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:25.008903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 491/972] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:25.010025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ed094-5297-4dfa-9568-ca9cd662de01 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.010134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ed094-5297-4dfa-9568-ca9cd662de01 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.010346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ed094-5297-4dfa-9568-ca9cd662de01 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.011682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 492/973] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:25.015805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ed094-5297-4dfa-9568-ca9cd662de01 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.026925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.027559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.027841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.028101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.028689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.030072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b2e0c4f9-2dda-4e61-868d-d4f2cef9ac31 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:25.039667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:25.040881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:25.041766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 482/974] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:25.042695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.042695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.043215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.060415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:25.061946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:25.119720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ed094-5297-4dfa-9568-ca9cd662de01 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.128265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 483/975] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:25 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:25.129228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.147449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.161202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.161827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.162122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.162390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.163017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:25.163343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.164972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.182035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 493/976] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:25.182997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.183808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.238359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772e9c3-be84-4f1e-b154-8c0ff05bfdab None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.273802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.274696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.275169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.275755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.276684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:25.277214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.278799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.283549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 484/977] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:25.285401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd98b97-f23b-4ea7-82ee-8ebe9bbca09a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.289217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.324293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.325171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.325868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.327043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.335848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.337607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:25.348969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:25.350091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 19:38:25.351372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:25.354895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 494/978] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:25.355703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.397984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf948aa-cdbc-4c0d-85fe-3699eec34092 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.398578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.411550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.432362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.432962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.433230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.433493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.434144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:25.434438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.435758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.440271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 485/979] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:25.440998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.458556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba124596-52fa-4a5f-8e64-fbd2d7861069 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.492330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.492925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.493200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.493468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.494211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:25.494508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.495913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.513217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 495/980] 10.222.0.15 () {56 vars in 1149 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:25.514023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.540299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737c7d0-72f4-44c0-8607-f65f298872fd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.573755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.574355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.574630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.574896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.575536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.576883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.592566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 486/981] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:25 2022] POST /identity/v3/services => generated 242 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:25.593495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.610704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.613999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5acc7a-cae1-47a2-864e-70e8d7df7d47 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.638520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.665316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.665972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.666252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.666536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.667240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] RBAC: Adding query filter params (interface=admin, service_id=9cfff7ff45a9483184584e566736811d, region_id=RegionOne) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:25.667557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.668953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.674634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 496/982] 10.222.0.15 () {56 vars in 1249 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/endpoints?service_id=9cfff7ff45a9483184584e566736811d&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:25.681877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:25.721555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8c8fca-2d0f-4cc1-aa84-a68f956a92d3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.763693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.765043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.765313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.765590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.795711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d13f4f7-3e25-4b0e-9d26-d6807eb932b0 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:25.832750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:25.836069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:25.836069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:25.836069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:25.836069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.836069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.842918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 497/983] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:38:25.850202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:25.851685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:25.900032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:25.972897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 487/984] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:25.974248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.002966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:26.082259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff256da-8017-4929-84ab-c97a1fb18239 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.117591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.118253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.118531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.118801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.128925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.130376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:26.144578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:26.145537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:26.150891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:26.150891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:26.150891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:26.150891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 498/985] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:26.152958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.251938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81500a69-909c-4ac6-a730-13ef9b5c6293 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.286140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.287607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.288071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.288525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.289330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:26.289818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.291340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.298227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:26.308972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 488/986] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:26.310237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.390526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:26.415262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c2b486-6e48-49a4-8d6e-f60aa6f4bc04 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.448848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.449479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.449776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.450050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.456713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.458156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:26.469987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:26.470895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:26.471809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 499/987] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:26.478763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.579002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fb55ea-0c4b-4b89-8cdd-67c40271583d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.612480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.613081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.613364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.613623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.614471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:26.614772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.616356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0587dd5c-e450-4292-bf99-2f9bc8a132ee None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.628323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 489/988] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:26.634756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:26.737896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f44728-5895-4629-bbc7-aa95c27a4372 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:26.771910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:26.772505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:26.772780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:26.773046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:26.773773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] RBAC: Adding query filter params (role.id=bf95583bb5cc4f7b80ea496902da57f4, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=fce06803dd124d6680a9ce5792e7ea90) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:26.774080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:26.775476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d6800-471c-4b8f-8f3b-7a10a7098a02 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:26.778227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:26.782543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 500/989] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:26 2022] GET /identity/v3/role_assignments?user.id=fce06803dd124d6680a9ce5792e7ea90&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=bf95583bb5cc4f7b80ea496902da57f4 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:27.803173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87b95a-4c11-451d-b435-a807cf92adb7 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:27.803732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87b95a-4c11-451d-b435-a807cf92adb7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:27.803732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd87b95a-4c11-451d-b435-a807cf92adb7 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:27.805176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 490/990] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:27.810219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5788a1-3a0e-4b77-ab87-4f195ed21241 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:27.810612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5788a1-3a0e-4b77-ab87-4f195ed21241 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:27.810815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5788a1-3a0e-4b77-ab87-4f195ed21241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:27.829709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-3a5788a1-3a0e-4b77-ab87-4f195ed21241 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:27.831269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3a5788a1-3a0e-4b77-ab87-4f195ed21241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:27.909785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 501/991] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:28.005448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779d1c-4a5b-45f5-b7be-2c0071210fcd None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.006254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779d1c-4a5b-45f5-b7be-2c0071210fcd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.006525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9779d1c-4a5b-45f5-b7be-2c0071210fcd None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.008121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 491/992] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:28.013172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.013467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.013758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.032526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:28.033721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:28.101116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 502/993] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:28.175947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9779d1c-4a5b-45f5-b7be-2c0071210fcd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.233527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:28.275099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9779d1c-4a5b-45f5-b7be-2c0071210fcd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.308266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.308876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.309147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.309406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.310004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.311336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:28.319810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:28.320802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:28.321638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:28.327312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 492/994] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services/compute => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:28.327442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.376226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.386501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:28.437989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d724a9c-1082-4b59-a04e-f2242a2fb7d7 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.472204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.472806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.473082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.473348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.473986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:28.474796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.476139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.485203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 503/995] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:28.486784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.494679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fc0df8-7974-483e-99b6-498a1558cad4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.519236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:28.532201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.532939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.533210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.533477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.534079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.535694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:28.546766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:28.547783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:28.548663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 493/996] 10.222.0.15 () {56 vars in 1126 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:28.554933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.595907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499491b6-5e1b-413b-8c88-213475c9bd11 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.622344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:28.634318 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.634860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.635577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.635577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.636025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] RBAC: Adding query filter params (type=compute) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:28.636325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.637686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.648067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 504/997] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:28.649977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.675092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854c7f-f7c6-48f7-b141-8937d44593f6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.715742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.716337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.716614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.716906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.717502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:28.717818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.719156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.723888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 494/998] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:28.729941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.766504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcab80-5001-40d4-a0b1-531ba157f051 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.780469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:28.802437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.803035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.803315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.803583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.804262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] RBAC: Adding query filter params (interface=public, service_id=71ba4eee202240db831a28978e1523aa, region_id=RegionOne) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:28.804566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.806010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.815657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 505/999] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/endpoints?service_id=71ba4eee202240db831a28978e1523aa&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:28.816660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.851904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18258845-53a9-49f3-82a5-1d5d7dcc8d90 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.876834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:28.890456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.890982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.891252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.891518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.892140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:28.892432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.893795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.899231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 495/1000] 10.222.0.15 () {56 vars in 1122 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:28.904510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:28.927911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494c8c4a-ad5c-460a-9b3e-05684925300a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:28.966860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d378a5f6-8984-45e8-a495-9d7e553ed935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:28.967776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378a5f6-8984-45e8-a495-9d7e553ed935 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.968055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378a5f6-8984-45e8-a495-9d7e553ed935 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.968323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d378a5f6-8984-45e8-a495-9d7e553ed935 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.968925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378a5f6-8984-45e8-a495-9d7e553ed935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:28.970376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d378a5f6-8984-45e8-a495-9d7e553ed935 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:28.976671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 506/1001] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:28 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:38:28.977822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ebe0a-2a44-4e9f-a902-c06f6ee808b7 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.978102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ebe0a-2a44-4e9f-a902-c06f6ee808b7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.978372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ebe0a-2a44-4e9f-a902-c06f6ee808b7 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:28.979543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 507/1002] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:28.991512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:28.991817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:28.992025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.012095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:29.013323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:29.028584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f1293d-f876-4f9e-96a7-f0c18e078147 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:29.045891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:29.077315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:29.077977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.078258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.078711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.079404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:29.080783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:29.093788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 508/1003] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:28 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:29.102767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 496/1004] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:28 2022] POST /identity/v3/endpoints => generated 325 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:29.372215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:29.421702 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:29.477342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32ae25-4d85-428a-a41c-d0c5cd5681bb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:29.518128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:29.518759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.519031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.519295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] PATH_INFO: `/v3/services/image` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.519863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:29.521382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:29.530180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:29.531167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:29.532604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 509/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Sep 13 19:38:29 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:29.537126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:29.643516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e9b98-c1b8-4ca2-a392-f512b5a0cb6d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:29.679145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:29.679743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.680016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.680282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.680903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] RBAC: Adding query filter params (name=image) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:29.681200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:29.682575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659954d8-55d7-4c08-8ec3-b5dcb6b38baf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:29.689725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 497/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:29 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:29.695219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:29.797877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c2da5-7b88-4367-9c46-134a99827e18 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:29.805746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:29.841314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:29.842024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:29.842241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:29.842512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:29.843147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] RBAC: Adding query filter params (type=image) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:29.843447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:29.844841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071864b7-ed90-4127-a357-82f33ebfcdd3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:29.856711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 510/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:29 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:31.203312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cfd5-2f92-4e23-adca-7bdec7aa2594 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.203808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cfd5-2f92-4e23-adca-7bdec7aa2594 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.203808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cfd5-2f92-4e23-adca-7bdec7aa2594 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.205231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 498/1008] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:31.210422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.210806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.210964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.229493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:31.230700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:31.285563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3791c1-ea2d-43c5-8c24-baba03d8b309 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.285935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3791c1-ea2d-43c5-8c24-baba03d8b309 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.286708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3791c1-ea2d-43c5-8c24-baba03d8b309 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.288045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 499/1009] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:31.292841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.293115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.293401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.298822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 511/1010] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:31.311727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:31.312890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:31.341341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:31.380389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 500/1011] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:31 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:31.573002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:31.654063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:31.671170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:31.675704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b47ae-37fc-420b-ac77-eb46167e509d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:31.711959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:31.712578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.712889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.713143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.713750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:31.715081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:31.726512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:31.727516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 19:38:31.728369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:31.729805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 512/1012] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:31.735546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:31.764279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5bf95-4314-476a-ab37-db6be17c60e8 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:31.802906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:31.803537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.803809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.804081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.805196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:31.806644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:31.810245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 501/1013] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:31.814523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:31.850358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c7e418-fbef-4481-b5d6-0ffc323c90c1 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:31.864544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:31.888022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:31.888631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.888912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.889338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.890003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] RBAC: Adding query filter params (name=compute) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:31.890302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:31.891759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:31.897349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 513/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:31 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:31.903044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:31.934898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:31.948286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d1256-bdd9-42d9-af2e-0124315d7757 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:31.987996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d0e4d1-a9c6-4467-9670-5f5a6a5a5b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:31.988599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0e4d1-a9c6-4467-9670-5f5a6a5a5b4e None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:31.988875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0e4d1-a9c6-4467-9670-5f5a6a5a5b4e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:31.989165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d0e4d1-a9c6-4467-9670-5f5a6a5a5b4e None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:31.990275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0e4d1-a9c6-4467-9670-5f5a6a5a5b4e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:31.991902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d0e4d1-a9c6-4467-9670-5f5a6a5a5b4e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:32.017144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10401d27-c020-445f-9c76-d7131d0dbe05 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:32.026083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 502/1015] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:31 2022] POST /identity/v3/projects => generated 311 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:38:32.054078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4792fe-67f1-466a-a7ba-3a8cc80a32ad None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.054427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4792fe-67f1-466a-a7ba-3a8cc80a32ad None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.054678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4792fe-67f1-466a-a7ba-3a8cc80a32ad None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.055350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:32.056174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.056283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 503/1016] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:32.056358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.056588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.057477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] RBAC: Adding query filter params (type=compute) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:32.057815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:32.059298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:32.062502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.062816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.063059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.070878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 514/1017] 10.222.0.15 () {56 vars in 1112 bytes} [Tue Sep 13 19:38:31 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:32.071827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:32.087576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:32.088883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:32.169193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 504/1018] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:32.182904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:32.194103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5fe722-1220-4481-9f48-e9ca62fb562c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:32.230657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65963e3c-d1a8-41d9-a99b-c7eb6acdc45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:32.231334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65963e3c-d1a8-41d9-a99b-c7eb6acdc45b None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.231612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65963e3c-d1a8-41d9-a99b-c7eb6acdc45b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.231882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65963e3c-d1a8-41d9-a99b-c7eb6acdc45b None admin] PATH_INFO: `/v3/endpoints` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.232607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65963e3c-d1a8-41d9-a99b-c7eb6acdc45b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:32.234122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65963e3c-d1a8-41d9-a99b-c7eb6acdc45b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:32.255930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 515/1019] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:32 2022] POST /identity/v3/endpoints => generated 334 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:32.456890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:32.475759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:32.562993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86a7af1-af70-4acd-8234-d7b54884c5b0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:32.603774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f256ef-044b-4a1e-8246-e16e631138a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:32.604379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f256ef-044b-4a1e-8246-e16e631138a6 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:32.604648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f256ef-044b-4a1e-8246-e16e631138a6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:32.604910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f256ef-044b-4a1e-8246-e16e631138a6 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:32.605545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f256ef-044b-4a1e-8246-e16e631138a6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:32.606919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f256ef-044b-4a1e-8246-e16e631138a6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:32.624945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 505/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:38:32 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:34.258152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f850c9-db6a-4d4b-984b-5b4e43216002 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.258674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f850c9-db6a-4d4b-984b-5b4e43216002 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.258757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f850c9-db6a-4d4b-984b-5b4e43216002 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.260167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 516/1021] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:34.265175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7dc96-af5b-4db5-a06a-73f9413d2ba7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.265569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7dc96-af5b-4db5-a06a-73f9413d2ba7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.265725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7dc96-af5b-4db5-a06a-73f9413d2ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.283551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0ab7dc96-af5b-4db5-a06a-73f9413d2ba7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:34.284732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0ab7dc96-af5b-4db5-a06a-73f9413d2ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:34.352444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:34.364868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 506/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:34.514055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3eff3-556d-4610-948a-c697c5f01358 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.514242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3eff3-556d-4610-948a-c697c5f01358 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.514499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc3eff3-556d-4610-948a-c697c5f01358 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.515978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 517/1023] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:34.521458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ccb94-afdb-4a89-91b6-e4d22fe45e81 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.521761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ccb94-afdb-4a89-91b6-e4d22fe45e81 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.522036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60ccb94-afdb-4a89-91b6-e4d22fe45e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.540509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c60ccb94-afdb-4a89-91b6-e4d22fe45e81 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:34.541691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c60ccb94-afdb-4a89-91b6-e4d22fe45e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:34.607231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 507/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:34.635462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3eff3-556d-4610-948a-c697c5f01358 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:34.752768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc3eff3-556d-4610-948a-c697c5f01358 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:34.781791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102fab3-9f53-427f-8e78-fe48be7b4fa8 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.781999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102fab3-9f53-427f-8e78-fe48be7b4fa8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.782271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b102fab3-9f53-427f-8e78-fe48be7b4fa8 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.783595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 508/1025] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:34.789294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.789576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.789782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:34.789921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.790412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.790689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.790957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.792064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:34.793476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:34.797948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 518/1026] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:34.800420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:34.810027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:34.811211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:34.820290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:34.889671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 509/1027] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:34 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:34.890527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:34.914367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d27a90f-6b43-4b4e-a7fd-5f871215aafd None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:34.949830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:34.951269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:34.952041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:34.952466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:34.952929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:34.953720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:34.955311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:34.978619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 519/1028] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:38:34 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:34.993916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f4ae3-29dc-48da-8bf4-d519d1569abd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.035676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.036710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.037184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.037689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.051952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:35.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:35.058036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:35.062878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 510/1029] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:35.064840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.141153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.158312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.179773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bbe8ac-0389-421e-8828-b400a13b38bb None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.220097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.220713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.221000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.221272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.221985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:35.222288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.223709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.233392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 520/1030] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:35.239890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.279552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99eea6-8bcf-4c0a-b83f-ddcf3980c4e5 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.302534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.319044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.319670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.319943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.320211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] PATH_INFO: `/v3/services/image` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.320793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.322204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:35.331846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:35.332931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:35.333815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 511/1031] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/services/image => generated 86 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:35.340410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.353074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f44b48-e13c-460f-809f-a4d5bc9e3328 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.388094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.388755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.389028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.389294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.391589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.401077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.402545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:35.416876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:35.417908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:35.418824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:35.418824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:35.418824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:35.418824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 521/1032] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:35.423612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.452208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4ef92-08ba-43ca-9909-fa11c34a2476 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.491300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.492217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.492491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.492753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.493374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] RBAC: Adding query filter params (name=image) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:35.493662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.495035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.500083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 512/1033] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:35.506166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.539706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1a3a3-5112-4a2d-b7eb-7ca858ab6a78 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.557969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.574145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.574746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.575021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.575288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.575916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:35.576214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.577571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.589301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 522/1034] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:35.594766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.619138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4240819-e9f4-4180-9204-a089352ddaee None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.640620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.661957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.663040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.663316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.663856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.664486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] RBAC: Adding query filter params (type=image) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:35.664789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.666164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.675682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 513/1035] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:35.677459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.702277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33877134-18a8-4ddc-9c9f-daff652fe173 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.737403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.738082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.738363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.740940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.746777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.748224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:35.760268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:35.761311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:35.762843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 523/1036] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:35.766968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.786320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee01bb3-9327-4fc8-a05f-1c9dabaa0cc8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.800634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.824694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.825334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.825612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.826035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.826741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] RBAC: Adding query filter params (interface=public, service_id=d0c1fa45597944de88ead08e9bda7c21, region_id=RegionOne) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:35.827043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.828431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.834807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 514/1037] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/endpoints?service_id=d0c1fa45597944de88ead08e9bda7c21&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:35.840255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.876666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dab8b2-ace0-49ae-9f5d-f9e228bc45d8 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.883879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:35.919594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.920281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.920557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.920826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.921623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:35.921976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.923508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:35.947083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 524/1038] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:35.948384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:35.954208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d50e49-6404-4c71-a0c2-15aba241040d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:35.990684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:35.991283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:35.991556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:35.991826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:35.992540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:35.994035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.006552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 515/1039] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/services => generated 1957 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:38:36.053985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e1625-67b3-4f40-90da-f352d277e946 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.091086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.091627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.091907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.092177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.124931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:36.166105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.167563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.268932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 525/1040] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:35 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:38:36.275400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:36.317848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:36.377667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6be81-16b4-46ec-8322-59cebff00493 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.411139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.411755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.412028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.412295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.423354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.424764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:36.438809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:36.439783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:36.440702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:36.440702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:36.440702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:36.440702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 516/1041] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:36.447236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:36.511145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:36.548703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981554e5-e7cf-49cc-bfff-4d8e740355c2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.582980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.583585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.583860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.584127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.584758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:36.585057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.586447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.598003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 526/1042] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:36.603385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:36.703959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:36.713994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9147116-6250-47f7-9c1f-749a671178d1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.747498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.748128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.748402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.748666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.755968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.757497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:36.768820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:36.770056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:36.770984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:36.774067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 517/1043] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:36.775713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:36.885205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec190fc3-844f-4b7d-9dd8-1efc34114429 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:36.909993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:36.921506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:36.922300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:36.922580 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:36.922850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:36.923531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:36.923833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:36.925249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b8f29-7fb5-4bac-96bc-46a81d3490df None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:36.938241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 527/1044] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:36.944589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:37.050213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c05ca0f-493e-4d1c-ac17-5503e073c900 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:37.084374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:37.084994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:37.085270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:37.085537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:37.086329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] RBAC: Adding query filter params (role.id=bf95583bb5cc4f7b80ea496902da57f4, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=910ca69bc9bf4f328d9537f0b9bb1658) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:37.086635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:37.088104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3d6143-13e5-496e-b934-7cc50ccf2ffc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:37.097857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 518/1045] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:36 2022] GET /identity/v3/role_assignments?user.id=910ca69bc9bf4f328d9537f0b9bb1658&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=bf95583bb5cc4f7b80ea496902da57f4 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:37.212066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce28c27-14dc-4da4-ab15-aafc83d27c0d None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:37.212304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce28c27-14dc-4da4-ab15-aafc83d27c0d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:37.212519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce28c27-14dc-4da4-ab15-aafc83d27c0d None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:37.214187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 528/1046] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:37.219994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:37.220286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:37.220546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:37.240663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:37.241908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:37.254757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:37.313607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 519/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:37.581554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce28c27-14dc-4da4-ab15-aafc83d27c0d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:37.683191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce28c27-14dc-4da4-ab15-aafc83d27c0d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:37.716890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:37.717522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:37.717824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:37.718098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:37.728880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:37.733620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:37.734719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:37.735587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:37.738055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 529/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:37.740662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:37.846940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f89cbed-8e30-4480-ab57-bb0ee6408f16 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:37.881442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:37.882072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:37.882345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:37.882614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:37.883270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:37.883567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:37.884903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:37.893087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 520/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:37.899231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:37.948002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:38.004427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c2895-b8f9-4ed0-a2ea-49e418aa5daa None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:38.038246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e08790-72ab-4bb9-8496-5c9c25577ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:38.038871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e08790-72ab-4bb9-8496-5c9c25577ccd None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.039149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e08790-72ab-4bb9-8496-5c9c25577ccd None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.039422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e08790-72ab-4bb9-8496-5c9c25577ccd None admin] PATH_INFO: `/v3/users/aa1e80fe63e249dea569b76a932e2871` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.052648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e08790-72ab-4bb9-8496-5c9c25577ccd None admin] RBAC: Authorizing `identity:get_user(user_id=aa1e80fe63e249dea569b76a932e2871)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:38.054102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e08790-72ab-4bb9-8496-5c9c25577ccd None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:38.062110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 530/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:37 2022] GET /identity/v3/users/aa1e80fe63e249dea569b76a932e2871 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:38.062271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:38.073844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:38.166532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0bb39b-e1da-4e75-9b51-cef3c169bb06 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:38.199949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:38.200553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.200828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.201098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.205539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e81fd-efae-4ea9-b600-84dda6ad12b1 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.205850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e81fd-efae-4ea9-b600-84dda6ad12b1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.206123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e81fd-efae-4ea9-b600-84dda6ad12b1 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.208209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 531/1051] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:38.213150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.213433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.213712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.232114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:38.233290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:38.275980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:38.277409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:38.287774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:38.305150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 532/1052] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:38 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:38.379809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 521/1053] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:38:38.385616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:38.417308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:38.489571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58342006-7006-41b7-b036-9e8f90d319fa None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:38.523555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:38.524177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.524449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.524718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.525836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] RBAC: Authorizing `identity:get_project(project_id=16fd2e3336294e48b71850d057955719)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:38.527245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:38.529022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 533/1054] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/projects/16fd2e3336294e48b71850d057955719 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:38.534790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:38.574427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:38.638642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a027fd-7d97-4abc-9df2-ac8705f2889c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:38.648277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:38.675416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:38.676812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.677082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.677345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.678092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=16fd2e3336294e48b71850d057955719, user.id=aa1e80fe63e249dea569b76a932e2871) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:38.678390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:38.679594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:38.680060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:38.682675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983e3a8a-3d28-429a-9f1d-4ae0ddffc278 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:38.693810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 522/1055] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/role_assignments?user.id=aa1e80fe63e249dea569b76a932e2871&scope.project.id=16fd2e3336294e48b71850d057955719&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:38.724378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:38.725033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.725310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.725580 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] PATH_INFO: `/v3/services/image` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.726203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:38.727575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:38.737073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:38.738856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:38.739723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 534/1056] 10.222.0.15 () {56 vars in 1105 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:38.744928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:38.858151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e3aaee-8689-459f-8a19-e4e6d6e0c3c5 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:38.894222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:38.894859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:38.895133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:38.895401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:38.896022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] RBAC: Adding query filter params (name=image) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:38.896319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:38.897834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:38.907395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 523/1057] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:38.908707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.010379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da9f3e-e3fc-4c98-8ed2-958853342aa3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.044358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.044969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.045245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.045513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.046168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] RBAC: Adding query filter params (type=image) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:39.046471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.047821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2d7cc5-2d74-4305-90be-27bf8ea3e030 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.053910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 535/1058] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:39.058954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.094113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.165426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e539ab60-8372-4f64-91be-9c4c990375dc None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.198734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135770ce-d4a2-4e5d-b99e-5ae35a5d0814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.199402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135770ce-d4a2-4e5d-b99e-5ae35a5d0814 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.199680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135770ce-d4a2-4e5d-b99e-5ae35a5d0814 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.199947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135770ce-d4a2-4e5d-b99e-5ae35a5d0814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.200619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135770ce-d4a2-4e5d-b99e-5ae35a5d0814 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.202006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135770ce-d4a2-4e5d-b99e-5ae35a5d0814 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.229766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 524/1059] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:38:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:39.230718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b348aa-4e32-41ef-9bc3-bd9c7368099d None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.231011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b348aa-4e32-41ef-9bc3-bd9c7368099d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.231288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b348aa-4e32-41ef-9bc3-bd9c7368099d None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.232889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 536/1060] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:39.238480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.238765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.239037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.258393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:39.259603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:39.341865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 525/1061] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:39.607007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b348aa-4e32-41ef-9bc3-bd9c7368099d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.636737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.711254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b348aa-4e32-41ef-9bc3-bd9c7368099d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.745128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.745778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.746060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.746329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.757308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:39.762081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:39.762987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:39.763837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:39.766047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 537/1062] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:39.769048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:39.787373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:39.877336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eb42d-eacb-4671-9d6e-0d65b1e555e4 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:39.911675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:39.912279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:39.912552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:39.912875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:39.913542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:39.913911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:39.915262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:39.923857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 526/1063] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:39.930151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.018557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:40.031411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93418801-2600-4b4c-a004-c77df56070f2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.065074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.065705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.066020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.066904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.077313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.078762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:40.092386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:40.093327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:40.094607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:40.094607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:40.094607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:40.098044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 538/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:40.099230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.184176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:40.202613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7203b-a350-4461-8c35-67f29c79d812 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.234873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.235463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.235733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.235997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.236624 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:40.236917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.238966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.250538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 527/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:40.255972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.356814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a985c945-614c-464d-8313-a9ff64d00104 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.390203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.390829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.391108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.391373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] PATH_INFO: `/v3/users/nova` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.398721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.400138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.402705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:40.413099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:40.414116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:40.415066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:40.418102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 539/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:40.419930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.521513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe47dfb2-c6bd-41c1-a00e-fbdee269886c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.555888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.556487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.556927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.557204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.557947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:40.558250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.559658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.568365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:40.583431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 528/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:40.584990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.687035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded2add-2179-40da-96ca-3056798d7f3a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.720413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.721115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.721325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.721569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.733477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.734973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-3ce6c325-bdc4-4e4f-8d55-739936de741b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:40.750776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:40.751824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:40.758108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:40.758108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:40.758108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:40.758108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 540/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:40.759897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.854082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df8818-7cf8-47bd-ad19-a68b1d8ad380 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.854382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df8818-7cf8-47bd-ad19-a68b1d8ad380 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.854663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df8818-7cf8-47bd-ad19-a68b1d8ad380 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.856113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 541/1069] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:40.860791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.861105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.861355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.869221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a26fb-38c5-43c7-a328-34f8b0cb3dc3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:40.880602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:40.882085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:40.908354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:40.908957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:40.909231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:40.909600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:40.910271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:40.910576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:40.911929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:40.923555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 529/1070] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:40.929274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:40.934952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:40.960012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 542/1071] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:38:40.988959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:41.032108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0f8e5-181d-434e-a117-99c76445f4fe None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.072302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.072939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.073204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.073471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.080199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:41.082228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:41.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:41.094619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:41.095635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 530/1072] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:41.100790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.199726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129ab69-5871-41d1-a7a3-86172e973574 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.233426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.244971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.245427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.245706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.246004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.246707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:41.247016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:41.248528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:41.269477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 543/1073] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:41.275801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.330251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:41.349671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a834b4-cd85-472e-92d3-71c988cfc4bf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.368870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:41.389439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c427e0e-68f2-4305-9618-8615b4fb4de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.390566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c427e0e-68f2-4305-9618-8615b4fb4de6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.391074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c427e0e-68f2-4305-9618-8615b4fb4de6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.391569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c427e0e-68f2-4305-9618-8615b4fb4de6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.402652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee5da1-624c-4bbb-85ec-f2b630aa31bc None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.403398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c427e0e-68f2-4305-9618-8615b4fb4de6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-4c427e0e-68f2-4305-9618-8615b4fb4de6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:41.408284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:41.409472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:41.410595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:41.412595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 531/1074] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:41.413968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1e4b3-1a13-40f2-85ee-97078802f866 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.414410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1e4b3-1a13-40f2-85ee-97078802f866 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.414830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1e4b3-1a13-40f2-85ee-97078802f866 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.416223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 532/1075] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:41.418989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1e4b3-1a13-40f2-85ee-97078802f866 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.440187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ec55e-6a67-4ca7-bbff-7424597e7afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.440929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec55e-6a67-4ca7-bbff-7424597e7afb None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.441750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec55e-6a67-4ca7-bbff-7424597e7afb None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.442402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ec55e-6a67-4ca7-bbff-7424597e7afb None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/910ca69bc9bf4f328d9537f0b9bb1658/roles/bf95583bb5cc4f7b80ea496902da57f4` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.445777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec55e-6a67-4ca7-bbff-7424597e7afb None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=910ca69bc9bf4f328d9537f0b9bb1658, role_id=bf95583bb5cc4f7b80ea496902da57f4)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:41.445777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ec55e-6a67-4ca7-bbff-7424597e7afb None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:41.460291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 544/1076] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:41 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/910ca69bc9bf4f328d9537f0b9bb1658/roles/bf95583bb5cc4f7b80ea496902da57f4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:38:41.461585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.461903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.462199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.482067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:41.483311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:41.546692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1e4b3-1a13-40f2-85ee-97078802f866 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.593774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 545/1077] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:41.594007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.594007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.594007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.594007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.594395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:41.594395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:41.594395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:41.604069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 533/1078] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:41.610053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.649064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:41.720151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a06afa-6f82-4ee0-a390-d8277862fb9d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.755009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.755635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.755898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.756167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] PATH_INFO: `/v3/users/aa1e80fe63e249dea569b76a932e2871` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.757271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] RBAC: Authorizing `identity:get_user(user_id=aa1e80fe63e249dea569b76a932e2871)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:41.758709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:41.760421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 546/1079] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:41 2022] GET /identity/v3/users/aa1e80fe63e249dea569b76a932e2871 => generated 285 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:41.766286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.809985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:41.869127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.878860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bbd074-becb-4b02-9d6f-c727a95ca032 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:41.915280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:41.915910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:41.916187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:41.916455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:41.917581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] RBAC: Authorizing `identity:get_project(project_id=16fd2e3336294e48b71850d057955719)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:41.919021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:41.926183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 534/1080] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:41 2022] GET /identity/v3/projects/16fd2e3336294e48b71850d057955719 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:41.930299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:41.983224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:41.986433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6f25d-e902-424b-bd1b-a92315e4ab4a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.024521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.025133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.025405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.025671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.033164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f5fbf-7af2-4cb5-8e41-bfde742b9117 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.044091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:42.070840 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.071488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.071751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.072017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719/users/aa1e80fe63e249dea569b76a932e2871/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.073987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] RBAC: Authorizing `identity:create_grant(project_id=16fd2e3336294e48b71850d057955719, user_id=aa1e80fe63e249dea569b76a932e2871, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:42.075352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:42.089795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 535/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:41 2022] PUT /identity/v3/projects/16fd2e3336294e48b71850d057955719/users/aa1e80fe63e249dea569b76a932e2871/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:38:42.096960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:42.099100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:42.205757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:42.210951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 547/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:42.216956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:42.322801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f3284-b7d2-4801-96e9-cd5eb7619bab None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.359668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.360300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.360572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.360837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.382716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:42.384194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-7c6c6e36-6d49-4d63-9e0b-d06bd3010211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:42.401261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:42.402528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:42.403439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:42.403439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:42.403439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:42.403439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 536/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:42.408536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:42.507691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65859f-5292-4084-a2c1-63146c8de3e2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:42.541902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:42.542502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:42.542779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:42.543048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:42.543680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:42.543979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:42.545324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e377cf-02eb-42ce-934d-9bc1055fbece None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:42.556862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 548/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:43.665233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f3f1-c322-4d98-bd84-ade2a4b767ce None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:43.665823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f3f1-c322-4d98-bd84-ade2a4b767ce None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:43.665823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0f3f1-c322-4d98-bd84-ade2a4b767ce None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:43.667134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 537/1085] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:43.672480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:43.672898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:43.673146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:43.691926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:43.693086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:43.746440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:43.759606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 549/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:44.027962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f3f1-c322-4d98-bd84-ade2a4b767ce None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.051308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:44.132413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0f3f1-c322-4d98-bd84-ade2a4b767ce None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.164651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b30ff-1d54-4f78-b716-ab6dbf594c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.165301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b30ff-1d54-4f78-b716-ab6dbf594c54 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.165575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b30ff-1d54-4f78-b716-ab6dbf594c54 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.165878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b30ff-1d54-4f78-b716-ab6dbf594c54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.179145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624b30ff-1d54-4f78-b716-ab6dbf594c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-624b30ff-1d54-4f78-b716-ab6dbf594c54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:44.184094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:44.185252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:44.186204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 538/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:44.191554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.288058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e84427-8899-4771-af4a-4562ec2c5562 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.288298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d73fcb-cd77-4947-b3a3-4bb98fbd9a9b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.288576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d73fcb-cd77-4947-b3a3-4bb98fbd9a9b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.288841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d73fcb-cd77-4947-b3a3-4bb98fbd9a9b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.290346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 539/1088] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:38:44.295162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.295870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.296148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.314181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:44.315371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:44.322131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.322784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.323064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.323329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.323979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:44.324276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:44.325613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:44.333885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 550/1089] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:44.339519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.379967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 540/1090] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:44.390306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:44.444994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abdd34b-1d60-4691-b075-9ea9f1f7da5e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.478624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.479250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.479532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.479806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.490089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:44.491507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:44.505105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:44.506264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:44.507406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:44.507406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:44.507406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:44.507406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 551/1091] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:44.511812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.524069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:44.607772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50086bd4-e3ef-4509-83f0-38e79e2171f0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.640365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.640955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.641231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.641503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.642166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:44.642469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:44.644414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:44.655308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 541/1092] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:44.660294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.660546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.730237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:44.767857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a64da-58f1-4157-8147-fa33c3993649 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.769922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41be5-e070-4f25-83e9-eb1fb758d5a9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.770757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:44.804109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.804357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.804737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.804974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.805070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.805268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.805374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.805651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.813329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:44.814910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:44.816600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:38:44.821283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:44.822465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:38:44.823339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 552/1093] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:38:44.828656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:38:44.829550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:44.835899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:44.837209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 542/1094] 10.222.0.15 () {56 vars in 1093 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:44.838116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.934414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf228c-b7dd-49e4-a6df-46bb4ecfb931 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.936767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a48ec4-bb42-4c89-b45c-e459fcb64d22 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:44.964535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:44.967354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.967969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.968245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.968519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.969200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:44.969479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:44.970873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:44.971308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:44.971946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:44.972211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:44.972471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:44.973242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:44.973539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:44.974944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:44.983932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 553/1095] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:44.984909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:44.997461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 543/1096] 10.222.0.15 () {56 vars in 1133 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:44.998969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.016412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.092660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a28990-5b9b-4ebc-a439-7faa5a590236 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.101656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3e86bf-16eb-45da-b77d-dddcbfd2df1d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.125631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.126281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.126562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.126834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] PATH_INFO: `/v3/users/aa1e80fe63e249dea569b76a932e2871` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.128019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] RBAC: Authorizing `identity:get_user(user_id=aa1e80fe63e249dea569b76a932e2871)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.129964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.131535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 554/1097] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/users/aa1e80fe63e249dea569b76a932e2871 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:45.137277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.137510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.138144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.138422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.138691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.192182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.204439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.205921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.226669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.234109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6ff2f-fecc-4d1e-b294-dfc7566d3148 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.267183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.267777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.268052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.268319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.302668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 544/1098] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:38:45.308290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.334166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.335761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.392044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.404276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8acd2df-4a32-4a40-9aac-6b8021e8528e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.436334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.438243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.438859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.439166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.439440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.444271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 555/1099] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:38:45.449503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.450118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.450954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:38:45.466341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:45.467440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:38:45.468529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:38:45.468529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:38:45.468529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:45.469892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 545/1100] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:45.475942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.546996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba0c2a7-c4b0-4280-9e74-064be857757d None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.571926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf374ad2-032c-4079-896e-3654d89756e3 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.580137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.580760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.581038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.581310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.582454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] RBAC: Authorizing `identity:get_project(project_id=16fd2e3336294e48b71850d057955719)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.583870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.585494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 556/1101] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/projects/16fd2e3336294e48b71850d057955719 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:45.591356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.604894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.605491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.605789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.606059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.606697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:45.606998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.608332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.618325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.621815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 546/1102] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:45.627268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.655732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.701872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f29fe-2a0a-4a85-a229-d23a4229f0c2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.724321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297be08-8eb7-4fc8-b70c-f74226cba9de None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.735103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.735698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.735974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.736243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.736953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=16fd2e3336294e48b71850d057955719, user.id=aa1e80fe63e249dea569b76a932e2871) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:45.737869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.739299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.744867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 557/1103] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/role_assignments?user.id=aa1e80fe63e249dea569b76a932e2871&scope.project.id=16fd2e3336294e48b71850d057955719&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:38:45.764112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.764761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.765035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.765299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.773459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.774951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:38:45.787752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:38:45.788887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:38:45.789888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 547/1104] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:38:45.799137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:45.910028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d90dad9-a2e6-4db2-9c1c-b82fc7a04b3a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:45.937163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:45.946918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:45.947527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:45.947803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:45.948074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:45.948769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:45.949069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:45.950556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca38b3-bf86-45e3-b502-0546b7edb325 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:45.963239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 558/1105] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:45.969301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:46.027085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:46.066814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8620c33-1611-45c1-b8b7-faf540dde075 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:46.099425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:46.100015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:46.100290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:46.100589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:46.101307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] RBAC: Adding query filter params (role.id=bf95583bb5cc4f7b80ea496902da57f4, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=910ca69bc9bf4f328d9537f0b9bb1658) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:46.101611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:46.103048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a57d7a-cbd0-4495-9011-e6ef9049ea25 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:46.108724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 548/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:45 2022] GET /identity/v3/role_assignments?user.id=910ca69bc9bf4f328d9537f0b9bb1658&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=bf95583bb5cc4f7b80ea496902da57f4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:47.899818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169ee68-3474-4bdf-becb-e5397678348e None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:47.900307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169ee68-3474-4bdf-becb-e5397678348e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:47.900400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7169ee68-3474-4bdf-becb-e5397678348e None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:47.901675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 559/1107] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:47.906807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163264-6397-4638-8c85-533795fc4281 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:47.907237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163264-6397-4638-8c85-533795fc4281 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:47.907355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96163264-6397-4638-8c85-533795fc4281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:47.926758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-96163264-6397-4638-8c85-533795fc4281 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:47.927947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-96163264-6397-4638-8c85-533795fc4281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:47.992928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 549/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:48.257366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169ee68-3474-4bdf-becb-e5397678348e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:48.354381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7169ee68-3474-4bdf-becb-e5397678348e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:48.387488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46ab8d-bbd3-4e1c-a19f-9e0ccc015498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:48.388097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ab8d-bbd3-4e1c-a19f-9e0ccc015498 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:48.388371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ab8d-bbd3-4e1c-a19f-9e0ccc015498 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:48.388638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46ab8d-bbd3-4e1c-a19f-9e0ccc015498 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:48.389722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46ab8d-bbd3-4e1c-a19f-9e0ccc015498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:48.391163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46ab8d-bbd3-4e1c-a19f-9e0ccc015498 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:48.392685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 560/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:38:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:48.398261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96163264-6397-4638-8c85-533795fc4281 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:48.451393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:48.497312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96163264-6397-4638-8c85-533795fc4281 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:48.529219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df49090-7c6d-4fe2-89f3-fdf06a887904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:48.529764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df49090-7c6d-4fe2-89f3-fdf06a887904 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:48.530038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df49090-7c6d-4fe2-89f3-fdf06a887904 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:48.530310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df49090-7c6d-4fe2-89f3-fdf06a887904 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:48.530975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df49090-7c6d-4fe2-89f3-fdf06a887904 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:48.532381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df49090-7c6d-4fe2-89f3-fdf06a887904 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:48.553208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 550/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:38:48 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:38:50.717933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468fd1-4e10-4e21-a801-ee3c7b1de307 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:50.718520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468fd1-4e10-4e21-a801-ee3c7b1de307 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:50.718520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468fd1-4e10-4e21-a801-ee3c7b1de307 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:50.719791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 561/1111] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:50.724921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:50.725340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:50.725460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:50.743230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:50.744397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:50.807944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 551/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:51.078314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468fd1-4e10-4e21-a801-ee3c7b1de307 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:51.132608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:51.176209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468fd1-4e10-4e21-a801-ee3c7b1de307 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:51.208479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:51.209086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:51.209362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:51.209632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] PATH_INFO: `/v3/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:51.211021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] RBAC: Authorizing `identity:get_role(role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:51.212360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:51.213625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 562/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:51 2022] GET /identity/v3/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:51.219268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:51.260160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:51.315685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57e0975-6676-439e-a3f9-06f17f190cf0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:51.347228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:51.347778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:51.348051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:51.348319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] PATH_INFO: `/v3/users/7cfd57441ee54f0c98dcc352b36b77b6` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:51.360750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] RBAC: Authorizing `identity:get_user(user_id=7cfd57441ee54f0c98dcc352b36b77b6)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:51.362687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:51.364229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 552/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:51 2022] GET /identity/v3/users/7cfd57441ee54f0c98dcc352b36b77b6 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:51.369899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:51.464627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b6d85-fecf-4bd9-91ec-c33e71ef7c92 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:51.488375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:51.499698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:51.500299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:51.500577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:51.500848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:51.564534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:51.566034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:51.659180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 563/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:51.665065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:51.757329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:51.761286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0b3e2c-2262-41c6-8b19-1b35fcb7af63 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:51.794644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6b8e4-c505-4001-bb44-a69543e2778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:51.795249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b8e4-c505-4001-bb44-a69543e2778b None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:51.795530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b8e4-c505-4001-bb44-a69543e2778b None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:51.795796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6b8e4-c505-4001-bb44-a69543e2778b None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:51.796881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b8e4-c505-4001-bb44-a69543e2778b None admin] RBAC: Authorizing `identity:get_project(project_id=16fd2e3336294e48b71850d057955719)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:51.798307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6b8e4-c505-4001-bb44-a69543e2778b None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:51.800156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 553/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:51 2022] GET /identity/v3/projects/16fd2e3336294e48b71850d057955719 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:51.806116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:51.828813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:51.902129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a582f-8543-4d9a-baae-bfac449d4893 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:51.935122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:51.935656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:51.935933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:51.936202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:51.936906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] RBAC: Adding query filter params (role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=16fd2e3336294e48b71850d057955719, user.id=7cfd57441ee54f0c98dcc352b36b77b6) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:51.937207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:51.938642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71f2e40-5174-402a-9369-24aede9febed None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:51.943849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 564/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:51 2022] GET /identity/v3/role_assignments?user.id=7cfd57441ee54f0c98dcc352b36b77b6&scope.project.id=16fd2e3336294e48b71850d057955719&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:54.033219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd652e1-ad62-4277-b690-2121f65d9e95 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.033772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd652e1-ad62-4277-b690-2121f65d9e95 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.033772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd652e1-ad62-4277-b690-2121f65d9e95 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.035109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 554/1118] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:54.040115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.040538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.040664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.059133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:54.060330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:54.124497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 565/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:54.392412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd652e1-ad62-4277-b690-2121f65d9e95 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:54.489651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd652e1-ad62-4277-b690-2121f65d9e95 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:54.521161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:54.521781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.522055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.522319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] PATH_INFO: `/v3/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.523673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] RBAC: Authorizing `identity:get_role(role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:54.525000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:54.526294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 555/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:54 2022] GET /identity/v3/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 223 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:54.531853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:54.550047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:54.629166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec39ffe-16a3-415a-873a-6f55eaa79b0a None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:54.661724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:54.662468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.662752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.663024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] PATH_INFO: `/v3/users/7cfd57441ee54f0c98dcc352b36b77b6` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.664159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] RBAC: Authorizing `identity:get_user(user_id=7cfd57441ee54f0c98dcc352b36b77b6)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:54.665567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:54.667199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 566/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:54 2022] GET /identity/v3/users/7cfd57441ee54f0c98dcc352b36b77b6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:54.672813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:54.705653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:54.769588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d172481e-363c-42bb-bb26-25aa868a74dd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:54.801878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0912e9a-8697-4846-a5a9-26d5fa6d4a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:54.802480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912e9a-8697-4846-a5a9-26d5fa6d4a6a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.802750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912e9a-8697-4846-a5a9-26d5fa6d4a6a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.803016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0912e9a-8697-4846-a5a9-26d5fa6d4a6a None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.804099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0912e9a-8697-4846-a5a9-26d5fa6d4a6a None admin] RBAC: Authorizing `identity:get_project(project_id=16fd2e3336294e48b71850d057955719)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:54.805491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0912e9a-8697-4846-a5a9-26d5fa6d4a6a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:54.807086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 556/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:54 2022] GET /identity/v3/projects/16fd2e3336294e48b71850d057955719 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:54.812759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:54.892209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:54.908553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0124aed-f6c0-44c0-8fac-b208f3017a70 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:54.940426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed37d81c-2dc7-4be3-87b8-d7f5018b8ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:54.941010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37d81c-2dc7-4be3-87b8-d7f5018b8ee1 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:54.941285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37d81c-2dc7-4be3-87b8-d7f5018b8ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:54.941554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed37d81c-2dc7-4be3-87b8-d7f5018b8ee1 None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719/users/7cfd57441ee54f0c98dcc352b36b77b6/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:54.943682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37d81c-2dc7-4be3-87b8-d7f5018b8ee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=16fd2e3336294e48b71850d057955719, user_id=7cfd57441ee54f0c98dcc352b36b77b6, role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:54.945099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed37d81c-2dc7-4be3-87b8-d7f5018b8ee1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:54.957869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 567/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:38:54 2022] PUT /identity/v3/projects/16fd2e3336294e48b71850d057955719/users/7cfd57441ee54f0c98dcc352b36b77b6/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:38:57.046924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fafec-4f72-4d2a-8d2c-a9ce20400b92 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.047447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fafec-4f72-4d2a-8d2c-a9ce20400b92 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.047447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fafec-4f72-4d2a-8d2c-a9ce20400b92 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.049563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 557/1124] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:38:57.054753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.055171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.055290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.073705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:38:57.074916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:38:57.137665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 568/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:38:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:38:57.403827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fafec-4f72-4d2a-8d2c-a9ce20400b92 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:57.501694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fafec-4f72-4d2a-8d2c-a9ce20400b92 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:57.503188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:57.535225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:57.535828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.536103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.536371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] PATH_INFO: `/v3/roles/65ca997309d242bca2f2272f2cbaaa2a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.537762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] RBAC: Authorizing `identity:get_role(role_id=65ca997309d242bca2f2272f2cbaaa2a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:57.539101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:57.540382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 558/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:57 2022] GET /identity/v3/roles/65ca997309d242bca2f2272f2cbaaa2a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:57.545977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:57.616749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:57.641295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44981e7-eb6f-43c5-bdb0-2661b2fceda0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:57.674306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:57.675269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.675759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.676255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] PATH_INFO: `/v3/users/7cfd57441ee54f0c98dcc352b36b77b6` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.678176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=7cfd57441ee54f0c98dcc352b36b77b6)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:57.680100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:57.681998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 569/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:38:57 2022] GET /identity/v3/users/7cfd57441ee54f0c98dcc352b36b77b6 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:57.689228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:57.785288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8aaf0-e5f6-4f39-bd41-878145c91daf None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:57.818118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:57.818705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:57.818977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:57.819244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:57.856893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:57.884032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:57.885443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:57.978299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 559/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:38:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:38:57.984192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:58.084601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170915dd-65cd-4833-b8bf-8e8f5f508cd0 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:58.117638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6375e9d-f1cc-49d8-8dbf-d78097cef8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:58.118284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6375e9d-f1cc-49d8-8dbf-d78097cef8a2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:58.118559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6375e9d-f1cc-49d8-8dbf-d78097cef8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:58.118829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6375e9d-f1cc-49d8-8dbf-d78097cef8a2 None admin] PATH_INFO: `/v3/projects/16fd2e3336294e48b71850d057955719` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:58.119929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6375e9d-f1cc-49d8-8dbf-d78097cef8a2 None admin] RBAC: Authorizing `identity:get_project(project_id=16fd2e3336294e48b71850d057955719)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:58.121349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6375e9d-f1cc-49d8-8dbf-d78097cef8a2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:58.122983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 570/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:38:57 2022] GET /identity/v3/projects/16fd2e3336294e48b71850d057955719 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:38:58.129551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:38:58.195327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:38:58.225310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae1567-8486-42d6-8ae8-4985fe091478 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:38:58.257683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:38:58.258288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:38:58.258558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:38:58.258821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:38:58.259520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] RBAC: Adding query filter params (role.id=65ca997309d242bca2f2272f2cbaaa2a, scope.project.id=16fd2e3336294e48b71850d057955719, user.id=7cfd57441ee54f0c98dcc352b36b77b6) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:38:58.259815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:38:58.261203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfea39a-9735-4a9e-be4b-4cdfef54ba7e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:38:58.266685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 560/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:38:58 2022] GET /identity/v3/role_assignments?user.id=7cfd57441ee54f0c98dcc352b36b77b6&scope.project.id=16fd2e3336294e48b71850d057955719&role.id=65ca997309d242bca2f2272f2cbaaa2a => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:00.412960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d6c72-70ac-4a01-85ca-818c7d1ff9cb None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:00.413528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d6c72-70ac-4a01-85ca-818c7d1ff9cb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:00.413528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d6c72-70ac-4a01-85ca-818c7d1ff9cb None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:00.414828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 571/1131] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:00.420115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:00.420547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:00.420650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:00.439159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:00.440319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:00.503904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 561/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:00.774298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d6c72-70ac-4a01-85ca-818c7d1ff9cb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:00.787029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:00.873497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d6c72-70ac-4a01-85ca-818c7d1ff9cb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:00.905866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6b659-3c4b-4ae8-9e04-ec8d19d135e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:00.906477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6b659-3c4b-4ae8-9e04-ec8d19d135e2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:00.906755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6b659-3c4b-4ae8-9e04-ec8d19d135e2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:00.907026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6b659-3c4b-4ae8-9e04-ec8d19d135e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:00.908673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6b659-3c4b-4ae8-9e04-ec8d19d135e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:00.910116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6b659-3c4b-4ae8-9e04-ec8d19d135e2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:00.911634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 572/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:00.917331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:00.976923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:01.012986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8813cf-f3b9-4b8e-91e4-dbcb8e8586e7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:01.046316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ca6005-ad6b-471e-aedb-a8c99318e05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:01.046851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6005-ad6b-471e-aedb-a8c99318e05d None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:01.047128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6005-ad6b-471e-aedb-a8c99318e05d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:01.047400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ca6005-ad6b-471e-aedb-a8c99318e05d None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:01.048079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6005-ad6b-471e-aedb-a8c99318e05d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:01.049785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ca6005-ad6b-471e-aedb-a8c99318e05d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:01.075236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 562/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:39:00 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:03.215800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7bbe8-02d7-4250-9a28-e72d36a5dbe9 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:03.216321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7bbe8-02d7-4250-9a28-e72d36a5dbe9 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:03.216321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7bbe8-02d7-4250-9a28-e72d36a5dbe9 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:03.217574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 573/1135] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:03.222745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:03.223145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:03.223283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:03.241434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:03.242731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:03.305639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 563/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:03.576871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7bbe8-02d7-4250-9a28-e72d36a5dbe9 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:03.652635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:03.675538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc7bbe8-02d7-4250-9a28-e72d36a5dbe9 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:03.706946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b5a11-a752-4b6e-b13c-2059e1a9daff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:03.707552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5a11-a752-4b6e-b13c-2059e1a9daff None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:03.707825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5a11-a752-4b6e-b13c-2059e1a9daff None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:03.708090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b5a11-a752-4b6e-b13c-2059e1a9daff None admin] PATH_INFO: `/v3/domains/default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:03.709171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b5a11-a752-4b6e-b13c-2059e1a9daff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:03.710599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b5a11-a752-4b6e-b13c-2059e1a9daff None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:03.712120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 574/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:39:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:03.717814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:03.762756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:03.813820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e99a1-b275-447c-b69a-6ccf5c33f02a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:03.846913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e320ab42-291f-4d39-a7e5-a19960b60808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:03.847506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320ab42-291f-4d39-a7e5-a19960b60808 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:03.847775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320ab42-291f-4d39-a7e5-a19960b60808 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:03.848037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e320ab42-291f-4d39-a7e5-a19960b60808 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:03.848679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320ab42-291f-4d39-a7e5-a19960b60808 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:03.850110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e320ab42-291f-4d39-a7e5-a19960b60808 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:03.870005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 564/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:39:03 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:06.015046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236bca-ceef-45cb-8dcd-dddba0a18585 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.015568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236bca-ceef-45cb-8dcd-dddba0a18585 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.015568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e236bca-ceef-45cb-8dcd-dddba0a18585 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:06.016922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 575/1139] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:06.023993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.024459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.024571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:06.042342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:06.043500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:06.107547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 565/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:06.373456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e236bca-ceef-45cb-8dcd-dddba0a18585 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:06.472131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e236bca-ceef-45cb-8dcd-dddba0a18585 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:06.504738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:06.505343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.505616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.505910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:06.515559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:06.520199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:06.521284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:39:06.522176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 576/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:06.527117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:06.557394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:06.623579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a31528-e2ce-4d7f-b129-547933f64c75 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:06.655030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:06.655611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.655880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.656142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:06.657282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:06.657571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:06.658937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:06.666783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 566/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:06.673574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:06.682755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:06.770763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4757c9-66fd-4f0b-8caf-03be4211f26b None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:06.803432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:06.804040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.804316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.804599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] PATH_INFO: `/v3/users/597a5790428b46aa8e5724ced573633f` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:06.816913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] RBAC: Authorizing `identity:get_user(user_id=597a5790428b46aa8e5724ced573633f)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:06.818361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:06.819905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 577/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:06 2022] GET /identity/v3/users/597a5790428b46aa8e5724ced573633f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:06.825660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:06.914767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:06.921897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78750a6-06e3-40f3-a4fb-1e69c29ed5b4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:06.953741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:06.954350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:06.954627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:06.955419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:07.019118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:07.020524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:07.107184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:07.114656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 567/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:07.120488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:07.181610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:07.218551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d97a75-daec-47eb-bb9c-7e2edaf37154 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:07.250887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c552daa8-066a-4b87-9720-43d5f71e3010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:07.251503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552daa8-066a-4b87-9720-43d5f71e3010 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:07.252299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552daa8-066a-4b87-9720-43d5f71e3010 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:07.252569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c552daa8-066a-4b87-9720-43d5f71e3010 None admin] PATH_INFO: `/v3/projects/f158bc01968740f1b9d4b44d34952ba8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:07.253675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552daa8-066a-4b87-9720-43d5f71e3010 None admin] RBAC: Authorizing `identity:get_project(project_id=f158bc01968740f1b9d4b44d34952ba8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:07.255110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c552daa8-066a-4b87-9720-43d5f71e3010 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:07.256733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 578/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:07 2022] GET /identity/v3/projects/f158bc01968740f1b9d4b44d34952ba8 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:07.263050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:07.357422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a87c9-bc8b-469b-a185-4e7242ae38eb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:07.390008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:07.390588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:07.390857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:07.391119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:07.391817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=f158bc01968740f1b9d4b44d34952ba8, user.id=597a5790428b46aa8e5724ced573633f) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:07.392112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:07.394069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb587d35-79de-4c6e-8090-03410dde348c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:07.399148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 568/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:07 2022] GET /identity/v3/role_assignments?user.id=597a5790428b46aa8e5724ced573633f&scope.project.id=f158bc01968740f1b9d4b44d34952ba8&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:09.501424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb56af4-3461-421b-9258-2bdf638b7905 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.502015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb56af4-3461-421b-9258-2bdf638b7905 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.502015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb56af4-3461-421b-9258-2bdf638b7905 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.503295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 579/1147] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:09.508360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.508774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.508887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.526480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:09.527642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:09.578694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:09.592281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 569/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:09.858088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb56af4-3461-421b-9258-2bdf638b7905 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:09.953719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb56af4-3461-421b-9258-2bdf638b7905 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:09.982089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:09.987581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:09.988191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:09.988464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:09.988732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:09.998435 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:10.003101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:10.004228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:39:10.005282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 580/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:10.010101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.105829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f46ac-e5c8-4aa8-b129-6410ef845196 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.138323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.138923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.139198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.139468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.140120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:10.140416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.141780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.150174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 570/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:10.156100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.252530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924f320-0599-4d77-a881-69d261335bd2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.285646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.286292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.286565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.286832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] PATH_INFO: `/v3/users/597a5790428b46aa8e5724ced573633f` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.287926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] RBAC: Authorizing `identity:get_user(user_id=597a5790428b46aa8e5724ced573633f)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.289334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.290908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 581/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/users/597a5790428b46aa8e5724ced573633f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:10.296868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.346173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:10.394970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a56e-d8ea-4801-80fd-0e124b02142a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.426800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3235145-ad62-42fe-8b6c-dcd907dcd1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.427404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235145-ad62-42fe-8b6c-dcd907dcd1dd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.427679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235145-ad62-42fe-8b6c-dcd907dcd1dd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.427948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3235145-ad62-42fe-8b6c-dcd907dcd1dd None admin] PATH_INFO: `/v3/projects/f158bc01968740f1b9d4b44d34952ba8` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.429036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3235145-ad62-42fe-8b6c-dcd907dcd1dd None admin] RBAC: Authorizing `identity:get_project(project_id=f158bc01968740f1b9d4b44d34952ba8)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.430466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3235145-ad62-42fe-8b6c-dcd907dcd1dd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.432058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 571/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:10 2022] GET /identity/v3/projects/f158bc01968740f1b9d4b44d34952ba8 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:10.437936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:10.496031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:10.535569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d249b4c-3430-4bc2-9b32-53322b85ff8a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:10.569054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7913f05-f39a-4e59-89b0-c56aa06073a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:10.569700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913f05-f39a-4e59-89b0-c56aa06073a0 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:10.570012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913f05-f39a-4e59-89b0-c56aa06073a0 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:10.570286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7913f05-f39a-4e59-89b0-c56aa06073a0 None admin] PATH_INFO: `/v3/projects/f158bc01968740f1b9d4b44d34952ba8/users/597a5790428b46aa8e5724ced573633f/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:10.572170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7913f05-f39a-4e59-89b0-c56aa06073a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=f158bc01968740f1b9d4b44d34952ba8, user_id=597a5790428b46aa8e5724ced573633f, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:10.573592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7913f05-f39a-4e59-89b0-c56aa06073a0 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:10.585882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 582/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:39:10 2022] PUT /identity/v3/projects/f158bc01968740f1b9d4b44d34952ba8/users/597a5790428b46aa8e5724ced573633f/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:39:12.713540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb9000-3ff4-43b7-bbbb-aeaca727613e None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:12.714161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb9000-3ff4-43b7-bbbb-aeaca727613e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:12.714161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb9000-3ff4-43b7-bbbb-aeaca727613e None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:12.715496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 572/1154] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:12.721167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:12.721594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:12.721694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:12.739228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:12.740419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:12.804109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 583/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:13.071046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb9000-3ff4-43b7-bbbb-aeaca727613e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:13.168484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb9000-3ff4-43b7-bbbb-aeaca727613e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:13.184601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:13.201225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:13.201854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.202131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.202400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.213257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:13.217971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:13.219110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:39:13.220273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 573/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:13.224977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:13.272547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:13.321998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09dd60-8949-4fd6-b4b4-6f424af2cfb5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:13.368063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:13.368835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.369109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.369375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.370066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:13.370362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:13.371708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:13.380803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 584/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:13.386974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:13.482644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b8d3b-432f-4b0f-8f9d-a89ac0f30e6e None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:13.514619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:13.515220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.515495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.515757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] PATH_INFO: `/v3/users/597a5790428b46aa8e5724ced573633f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.516858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] RBAC: Authorizing `identity:get_user(user_id=597a5790428b46aa8e5724ced573633f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:13.518272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:13.519826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 574/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:13 2022] GET /identity/v3/users/597a5790428b46aa8e5724ced573633f => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:13.525517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:13.621114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc23341-7052-4e1b-8a56-c7e11c355d15 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:13.633577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:13.653504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:13.654122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.654397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.654662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.719637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:13.721054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:13.814172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 585/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:13.820013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:13.865860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:13.915492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c8d96-066a-4d05-b3b6-bfd50f9ef060 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:13.947725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f868a16-2aa9-43f7-8cc2-4bb0ac0d415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:13.948326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f868a16-2aa9-43f7-8cc2-4bb0ac0d415e None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:13.948594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f868a16-2aa9-43f7-8cc2-4bb0ac0d415e None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:13.948854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f868a16-2aa9-43f7-8cc2-4bb0ac0d415e None admin] PATH_INFO: `/v3/projects/f158bc01968740f1b9d4b44d34952ba8` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:13.949956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f868a16-2aa9-43f7-8cc2-4bb0ac0d415e None admin] RBAC: Authorizing `identity:get_project(project_id=f158bc01968740f1b9d4b44d34952ba8)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:13.951346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f868a16-2aa9-43f7-8cc2-4bb0ac0d415e None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:13.952919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 575/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:13 2022] GET /identity/v3/projects/f158bc01968740f1b9d4b44d34952ba8 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:13.959619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:13.973231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:14.056451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516226dd-a2fd-41a3-a351-91e9bc0e9f2f None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:14.090160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:14.090751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:14.091027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:14.091295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:14.092005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=f158bc01968740f1b9d4b44d34952ba8, user.id=597a5790428b46aa8e5724ced573633f) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:14.092307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:14.093714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a4ee-ef34-40f8-a80a-a1f371dbe197 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:14.099291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 586/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:13 2022] GET /identity/v3/role_assignments?user.id=597a5790428b46aa8e5724ced573633f&scope.project.id=f158bc01968740f1b9d4b44d34952ba8&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:16.227316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae85da0-bf4f-4a08-97f1-0db72d193df7 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:16.227835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae85da0-bf4f-4a08-97f1-0db72d193df7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:16.227835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae85da0-bf4f-4a08-97f1-0db72d193df7 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:16.229093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 576/1162] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:16.234166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:16.234539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:16.234708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:16.253122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:16.254320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:16.321277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 587/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:16.587739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae85da0-bf4f-4a08-97f1-0db72d193df7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:16.685782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae85da0-bf4f-4a08-97f1-0db72d193df7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:16.698617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:16.718367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:16.718950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:16.719808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:16.720082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:16.783380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:16.784791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:16.878359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 577/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:16.884229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:16.892668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:16.980347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd27de-b61c-4f05-9687-b528e2f3dfc0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:17.013036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:17.013654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:17.013963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:17.014236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:17.021370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:17.022826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-9ea92790-9b97-4886-8af8-db690a5f6925 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:17.034523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:17.035737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:39:17.036816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 588/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:39:16 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:17.041588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:17.056253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:17.135595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b825e59-ec80-4b7c-87ec-29ced1777185 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:17.167494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:17.168073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:17.168344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:17.168609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:17.169226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:17.169518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:17.170873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9730968-e7a2-4ae9-918d-ccfa7744ec78 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:17.177920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 578/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Tue Sep 13 19:39:17 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:19.265507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ec45-c04c-4e41-b86c-4b29ef8407e8 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.266649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ec45-c04c-4e41-b86c-4b29ef8407e8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.266746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785ec45-c04c-4e41-b86c-4b29ef8407e8 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.268031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 589/1167] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:19.273227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9138ab-7b3a-4e27-ad78-b7de0c75ef32 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.273641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9138ab-7b3a-4e27-ad78-b7de0c75ef32 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.273797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9138ab-7b3a-4e27-ad78-b7de0c75ef32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.291648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4d9138ab-7b3a-4e27-ad78-b7de0c75ef32 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:19.292829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4d9138ab-7b3a-4e27-ad78-b7de0c75ef32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:19.360838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 579/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:19.631959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785ec45-c04c-4e41-b86c-4b29ef8407e8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:19.696093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:19.729897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785ec45-c04c-4e41-b86c-4b29ef8407e8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:19.762591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a9ca7-7aaa-4e22-a068-c235792ab78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:19.763178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a9ca7-7aaa-4e22-a068-c235792ab78c None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:19.763453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a9ca7-7aaa-4e22-a068-c235792ab78c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:19.763722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a9ca7-7aaa-4e22-a068-c235792ab78c None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:19.764368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a9ca7-7aaa-4e22-a068-c235792ab78c None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:19.765751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9a9ca7-7aaa-4e22-a068-c235792ab78c None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:19.780724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 590/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Tue Sep 13 19:39:19 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:21.930064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c4717-e7b8-413c-8222-ae6a239328e0 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:21.930572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c4717-e7b8-413c-8222-ae6a239328e0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:21.930658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c4717-e7b8-413c-8222-ae6a239328e0 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:21.931943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 580/1170] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:21.937030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:21.937508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:21.937612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:21.955619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:21.956779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:22.019944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 591/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:22.291104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c4717-e7b8-413c-8222-ae6a239328e0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:22.302162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:22.391459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c4717-e7b8-413c-8222-ae6a239328e0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:22.424331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626808c6-4899-47ba-8e74-92dcfc89a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:22.424943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626808c6-4899-47ba-8e74-92dcfc89a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:22.425216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626808c6-4899-47ba-8e74-92dcfc89a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:22.425485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626808c6-4899-47ba-8e74-92dcfc89a1c6 None admin] PATH_INFO: `/v3/domains/9afde31890844e7399bd953a75da17cb` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:22.435476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626808c6-4899-47ba-8e74-92dcfc89a1c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9afde31890844e7399bd953a75da17cb)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:22.436888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626808c6-4899-47ba-8e74-92dcfc89a1c6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:22.438991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 581/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Sep 13 19:39:22 2022] GET /identity/v3/domains/9afde31890844e7399bd953a75da17cb => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:22.444826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:22.499641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:22.542416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd32307d-32ac-4582-91bc-d54552d540e2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:22.575295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaedf99-0c5e-40ff-b4bd-e0a7c845dabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:22.575893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaedf99-0c5e-40ff-b4bd-e0a7c845dabf None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:22.576167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaedf99-0c5e-40ff-b4bd-e0a7c845dabf None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:22.576437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaedf99-0c5e-40ff-b4bd-e0a7c845dabf None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:22.577070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaedf99-0c5e-40ff-b4bd-e0a7c845dabf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:22.579147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaedf99-0c5e-40ff-b4bd-e0a7c845dabf None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:22.607479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 592/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:39:22 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:24.752688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd439aa-5087-42a7-acac-20df4d01d5fe None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.753278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd439aa-5087-42a7-acac-20df4d01d5fe None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.753278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd439aa-5087-42a7-acac-20df4d01d5fe None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.754571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 582/1174] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:24.759806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:24.760179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:24.760358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:24.778615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:24.779821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:24.844442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 593/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:25.110677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd439aa-5087-42a7-acac-20df4d01d5fe None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:25.175338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:25.208484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd439aa-5087-42a7-acac-20df4d01d5fe None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:25.239427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fdecb0-e5be-40cd-9aec-7e6ae0f0fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:25.240027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdecb0-e5be-40cd-9aec-7e6ae0f0fd60 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:25.240298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdecb0-e5be-40cd-9aec-7e6ae0f0fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:25.240560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fdecb0-e5be-40cd-9aec-7e6ae0f0fd60 None admin] PATH_INFO: `/v3/domains/9afde31890844e7399bd953a75da17cb` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:25.241641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdecb0-e5be-40cd-9aec-7e6ae0f0fd60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9afde31890844e7399bd953a75da17cb)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:25.243058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fdecb0-e5be-40cd-9aec-7e6ae0f0fd60 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.244573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 583/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Tue Sep 13 19:39:25 2022] GET /identity/v3/domains/9afde31890844e7399bd953a75da17cb => generated 267 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:25.250789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:25.288533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:25.346300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01f3b1-8a32-4577-8003-70ecf7846de5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:25.378549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9b49-8791-4d75-9bad-dc7604646ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:25.379072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9b49-8791-4d75-9bad-dc7604646ff8 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:25.379348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9b49-8791-4d75-9bad-dc7604646ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:25.379617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9b49-8791-4d75-9bad-dc7604646ff8 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:25.380248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9b49-8791-4d75-9bad-dc7604646ff8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:25.381665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9b49-8791-4d75-9bad-dc7604646ff8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:25.402006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 594/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:39:25 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:39:27.569813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce3a05-510e-4a41-b188-e1a1175130ef None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:27.570377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce3a05-510e-4a41-b188-e1a1175130ef None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:27.570377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ce3a05-510e-4a41-b188-e1a1175130ef None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:27.571717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 584/1178] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:27.576802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:27.577207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:27.578078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:27.595817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:27.597032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:27.661144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 595/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:27.929884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce3a05-510e-4a41-b188-e1a1175130ef None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:28.027443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ce3a05-510e-4a41-b188-e1a1175130ef None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:28.059681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:28.060290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.060565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.060836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.064007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:28.072418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:28.077133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:28.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:39:28.079072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 585/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:28.084217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:28.107977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:28.180386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84920613-a4cd-43ea-ba1b-1f535392e75c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:28.211913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:28.212507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.212783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.213053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.213704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:28.214033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:28.215380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:28.223455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 596/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:28.229263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:28.326323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f0a0a-adc7-422d-a8d7-8f04a6e37ab0 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:28.368206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:28.368743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.369019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.369287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] PATH_INFO: `/v3/users/1135b2482ada422584b467d7f956f1cb` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.381912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] RBAC: Authorizing `identity:get_user(user_id=1135b2482ada422584b467d7f956f1cb)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:28.383327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:28.384904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 586/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:28 2022] GET /identity/v3/users/1135b2482ada422584b467d7f956f1cb => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:28.390707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:28.473137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:28.487760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01871d-f044-44da-a000-b4966486f604 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:28.519994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:28.520589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.520867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.521134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.587689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:28.589125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:28.668853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:28.689804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 597/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:28.695741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:28.748886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:28.791656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3012c5-40af-4eb9-a842-de8d4f0833c4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:28.823366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7566a455-7ecb-4ee5-b0d3-81e81933b7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:28.823971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566a455-7ecb-4ee5-b0d3-81e81933b7a5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.824244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566a455-7ecb-4ee5-b0d3-81e81933b7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.824516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7566a455-7ecb-4ee5-b0d3-81e81933b7a5 None admin] PATH_INFO: `/v3/projects/2656cdcae7c641959d258fe5e868486f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.825623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566a455-7ecb-4ee5-b0d3-81e81933b7a5 None admin] RBAC: Authorizing `identity:get_project(project_id=2656cdcae7c641959d258fe5e868486f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:28.827058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7566a455-7ecb-4ee5-b0d3-81e81933b7a5 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:28.829180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 587/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:28 2022] GET /identity/v3/projects/2656cdcae7c641959d258fe5e868486f => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:28.835292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:28.931297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5e9a4-4db7-4598-a474-5c49e8b590c3 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:28.966131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:28.966724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:28.966999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:28.967267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:28.967976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=2656cdcae7c641959d258fe5e868486f, user.id=1135b2482ada422584b467d7f956f1cb) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:28.968277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:28.969679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0a1e11-d4a5-4149-ab5e-a4cf395fe613 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:28.974835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 598/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:28 2022] GET /identity/v3/role_assignments?user.id=1135b2482ada422584b467d7f956f1cb&scope.project.id=2656cdcae7c641959d258fe5e868486f&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:31.096330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548fe15-2ed9-4b67-a284-4bdac2ce2879 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:31.096829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548fe15-2ed9-4b67-a284-4bdac2ce2879 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:31.096829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a548fe15-2ed9-4b67-a284-4bdac2ce2879 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:31.098175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 588/1186] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:31.103198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:31.103621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:31.103735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:31.121912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:31.123079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:31.166362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:31.187094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 599/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:31.461298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548fe15-2ed9-4b67-a284-4bdac2ce2879 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:31.559251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a548fe15-2ed9-4b67-a284-4bdac2ce2879 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:31.581980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:31.593963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:31.594590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:31.594864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:31.595131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:31.605024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:31.609752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:31.610836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:39:31.611896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 589/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:31.616865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:31.716808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e99a4-cb7d-4aee-b77e-db318b1b474d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:31.749338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:31.749959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:31.750234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:31.750502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:31.751156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:31.751452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:31.752806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:31.760477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 600/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:31.766770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:31.861689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117435ae-f15b-4f49-a8d7-b5fde4fa4339 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:31.894416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:31.895020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:31.895291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:31.895554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] PATH_INFO: `/v3/users/1135b2482ada422584b467d7f956f1cb` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:31.896644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] RBAC: Authorizing `identity:get_user(user_id=1135b2482ada422584b467d7f956f1cb)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:31.898062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:31.899596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 590/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:31 2022] GET /identity/v3/users/1135b2482ada422584b467d7f956f1cb => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:31.905422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:31.947059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:32.003004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996b9670-dbf4-4bb8-b4f5-3aefba0416ab None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:32.035376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9417a94-be43-4b47-b757-6f25d1163302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:32.035992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9417a94-be43-4b47-b757-6f25d1163302 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:32.036268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9417a94-be43-4b47-b757-6f25d1163302 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:32.036538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9417a94-be43-4b47-b757-6f25d1163302 None admin] PATH_INFO: `/v3/projects/2656cdcae7c641959d258fe5e868486f` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:32.037634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9417a94-be43-4b47-b757-6f25d1163302 None admin] RBAC: Authorizing `identity:get_project(project_id=2656cdcae7c641959d258fe5e868486f)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:32.039077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9417a94-be43-4b47-b757-6f25d1163302 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:32.040676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 601/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:31 2022] GET /identity/v3/projects/2656cdcae7c641959d258fe5e868486f => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:32.046908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:32.094926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:32.143066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa94fa-cbb0-413a-98b1-c85f53dbf0af None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:32.176069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3307c001-832c-4880-8045-613988bbe9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:32.176700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307c001-832c-4880-8045-613988bbe9cc None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:32.176970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307c001-832c-4880-8045-613988bbe9cc None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:32.177235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307c001-832c-4880-8045-613988bbe9cc None admin] PATH_INFO: `/v3/projects/2656cdcae7c641959d258fe5e868486f/users/1135b2482ada422584b467d7f956f1cb/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:32.179093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307c001-832c-4880-8045-613988bbe9cc None admin] RBAC: Authorizing `identity:create_grant(project_id=2656cdcae7c641959d258fe5e868486f, user_id=1135b2482ada422584b467d7f956f1cb, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:32.180500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3307c001-832c-4880-8045-613988bbe9cc None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:32.192995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 591/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:39:32 2022] PUT /identity/v3/projects/2656cdcae7c641959d258fe5e868486f/users/1135b2482ada422584b467d7f956f1cb/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:39:34.300092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855689b6-2b3a-4c35-8037-0a752e553a94 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.300651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855689b6-2b3a-4c35-8037-0a752e553a94 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.300651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855689b6-2b3a-4c35-8037-0a752e553a94 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.301984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 602/1193] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:39:34.307199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.307570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.307740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.325725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:39:34.327519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:39:34.390208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 592/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:39:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:39:34.660692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855689b6-2b3a-4c35-8037-0a752e553a94 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.759106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855689b6-2b3a-4c35-8037-0a752e553a94 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.768741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.791484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.792090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.792361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.792625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.803576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:39:34.808278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:39:34.809605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:39:34.810545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 603/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:39:34.819384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:34.862647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:34.922839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78771f-23a2-45c6-ad84-bd6568cf2dc7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:34.955176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:34.955773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:34.956049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:34.956320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:34.956973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:34.957272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:34.958654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:34.966376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 593/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:34.972825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.067435 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6b32d-8493-4711-8ee7-f9dd2317ded8 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.099181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.099785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.100063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.100334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] PATH_INFO: `/v3/users/1135b2482ada422584b467d7f956f1cb` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.101422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] RBAC: Authorizing `identity:get_user(user_id=1135b2482ada422584b467d7f956f1cb)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.102854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.104373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 604/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Tue Sep 13 19:39:34 2022] GET /identity/v3/users/1135b2482ada422584b467d7f956f1cb => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:35.110583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.205108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f8691-780f-4003-afd7-c63b06aa7929 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.206543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:35.237674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.238278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.238551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.238816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.302467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.303869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.395835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 594/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:39:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:39:35.401601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.440011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:35.498411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94024e6e-d71c-4ba6-90f8-57663932ce01 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.530696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716eeced-95b6-4348-9570-d4ec6b33b150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.531298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eeced-95b6-4348-9570-d4ec6b33b150 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.531572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eeced-95b6-4348-9570-d4ec6b33b150 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.531838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716eeced-95b6-4348-9570-d4ec6b33b150 None admin] PATH_INFO: `/v3/projects/2656cdcae7c641959d258fe5e868486f` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.532925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eeced-95b6-4348-9570-d4ec6b33b150 None admin] RBAC: Authorizing `identity:get_project(project_id=2656cdcae7c641959d258fe5e868486f)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.534482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716eeced-95b6-4348-9570-d4ec6b33b150 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.536053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 605/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Tue Sep 13 19:39:35 2022] GET /identity/v3/projects/2656cdcae7c641959d258fe5e868486f => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:35.542010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:39:35.544101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:35.636347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db61f5-3c62-417b-a0ef-0078076d7ed7 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:39:35.670248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:39:35.671071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:39:35.671345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:39:35.671610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:39:35.672313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=2656cdcae7c641959d258fe5e868486f, user.id=1135b2482ada422584b467d7f956f1cb) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:39:35.672608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:39:35.674025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986d1d17-ccbc-4b67-a829-f79bccf1239c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:39:35.679425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 595/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:39:35 2022] GET /identity/v3/role_assignments?user.id=1135b2482ada422584b467d7f956f1cb&scope.project.id=2656cdcae7c641959d258fe5e868486f&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:39:45.526169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=47 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:39:45.676981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=67 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:31.006104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc4b9-c88b-468f-8d82-8208e182671a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:31.006783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc4b9-c88b-468f-8d82-8208e182671a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:31.006783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72dc4b9-c88b-468f-8d82-8208e182671a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:31.008247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 606/1201] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:31.014660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:31.015217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:31.015217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:31.035504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:31.036733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:31.116452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 596/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:31.425097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dc4b9-c88b-468f-8d82-8208e182671a None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:31.557050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72dc4b9-c88b-468f-8d82-8208e182671a None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:31.600407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:31.601099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:31.601336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:31.601612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:31.613489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:31.615360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:31.635746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:31.636849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:31.637870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:31.637870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:31.637870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:31.637870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 607/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:31.643636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:31.762994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870aef1-d13c-40e1-8df4-43ff3390c42a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:31.798494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:31.806455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:31.807064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:31.807403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:31.807677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:31.808328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:31.808674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:31.810139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd85c66-a3d2-4287-9a63-ae0d281e5830 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:31.831577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 597/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:31.832357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:31.906364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:31.971807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5781f-ac6c-4055-94c1-2208c72cd518 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:32.023290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ed7331-0f34-4f7e-b343-a096e2f85841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:32.025227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed7331-0f34-4f7e-b343-a096e2f85841 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:32.025504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed7331-0f34-4f7e-b343-a096e2f85841 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:32.025790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ed7331-0f34-4f7e-b343-a096e2f85841 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:32.026539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed7331-0f34-4f7e-b343-a096e2f85841 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:32.027955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ed7331-0f34-4f7e-b343-a096e2f85841 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:32.069760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 608/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Tue Sep 13 19:40:31 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:40:34.577633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0681-700b-457f-8d6f-28d0686bd60b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.579586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0681-700b-457f-8d6f-28d0686bd60b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.579586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0681-700b-457f-8d6f-28d0686bd60b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.579912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 598/1206] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:34.586285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:34.586789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:34.586953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:34.607489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:34.608882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:34.694093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 609/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:40:35.008840 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0681-700b-457f-8d6f-28d0686bd60b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.130546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0681-700b-457f-8d6f-28d0686bd60b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.174756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.175388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.175659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.175924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.190754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:35.196819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:35.198057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:35.198945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 599/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:35.204817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.253614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:35.333075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3b2d0-5cf2-4207-a8ab-998c3efbaf5e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.372100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.372718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.372989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.373310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.373942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:35.374246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:35.375847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:35.385937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 610/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:35.391789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.454256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:35.513880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337282f0-a1b8-4f07-b05e-63b2ab2d5a2f None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.563019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.563769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.564143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.564430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.580213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:35.581960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:35.599806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:35.600872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:35.601908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:35.601908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:35.601908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:35.601908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 600/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:35.607176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.726959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:35.730788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0b4c32-c04d-4dca-93fe-76e2605cf729 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.767909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.768685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.769033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.769333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.770003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:35.770310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:35.771669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:35.784524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 611/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:35.790833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:35.920269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbffe9-41a9-4db9-a080-8eb88db9e978 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:35.920885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:35.955148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:35.955148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:35.955456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:35.955648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] PATH_INFO: `/v3/users/placement` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:35.963250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:35.964672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:35.975998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:35.978731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:35.979997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 601/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:35.983325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:36.087989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0238ef-8b38-4be3-a5eb-423cee8a7bc4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:36.124545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:36.125148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:36.125472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:36.125761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:36.126482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:36.126784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:36.128282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:36.151203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 612/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:36.152433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:36.262565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4ca45b-8bc9-4ca6-bc2b-f923a594065a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:36.303269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:36.303968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:36.304315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:36.304677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:36.343937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:36.386628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:36.388057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:36.501780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 602/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:40:36.505138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:36.529693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:36.626386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f828f968-378f-447c-a995-7b981de9a425 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:36.665461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:36.666149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:36.666424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:36.666689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:36.678717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:36.680225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:36.696857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:36.698127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:36.699340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:36.699340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:36.699340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:36.705767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 613/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:36.706000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:36.780179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:36.823044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd2cf1b-b1b4-433e-b86d-a62b5e838b01 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:36.859236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:36.859830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:36.860104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:36.860368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:36.860997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:36.861292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:36.862669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:36.873976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 603/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:36.879821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:36.955660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:36.987773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d214de58-3b21-4805-a7fe-61dba96b1e86 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:37.026167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:37.026864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:37.027258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:37.027611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:37.036169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:37.037847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:37.049817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:37.050946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:37.052088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 614/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:37.058818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:37.166667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c294e9-1006-4ff4-8f24-b3a9af15acf4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:37.190773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:37.204539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:37.205231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:37.205617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:37.206027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:37.206945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:37.207317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:37.208948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13b471-034a-4ed6-8f82-fb4b6572759f None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:37.223004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 604/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:37.229050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:37.350510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a5710-5bf0-40a0-bb03-55cfd9db1c61 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:37.381829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:37.389875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:37.390516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:37.390900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:37.391280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:37.392123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=1f3be656bf6446f99a23910cd833d799) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:37.392540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:37.394251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c461cdaa-174e-43f1-8c49-afd706d7a613 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:37.400777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 615/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:37 2022] GET /identity/v3/role_assignments?user.id=1f3be656bf6446f99a23910cd833d799&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:39.553394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e32de-0d63-4e12-9f4d-ef7cf34aadf0 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:39.554013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e32de-0d63-4e12-9f4d-ef7cf34aadf0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:39.554013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e32de-0d63-4e12-9f4d-ef7cf34aadf0 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:39.555501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 605/1220] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:39.561575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:39.562210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:39.562210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:39.580171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:39.581427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:39.651723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 616/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:40:39.925334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e32de-0d63-4e12-9f4d-ef7cf34aadf0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.048047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e32de-0d63-4e12-9f4d-ef7cf34aadf0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.091739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.092534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.092919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.093245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.105340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:40.112096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:40.113078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:40.113948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 606/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:40:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:40.120092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.227289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5905936f-099d-4197-a71b-39265ec22c6f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.257436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:40.269795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.270399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.270680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.270963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.271636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] RBAC: Adding query filter params (name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:40.272026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:40.273345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:40.282084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 617/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:40:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:40.288288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.339526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:40.400801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b348b-9ad1-417f-bd8f-94a671d86ab4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.438661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.439619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.440109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.440580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.452440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:40.453908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:40.468868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:40.470076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:40.471332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:40.471332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:40.471332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:40.471332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 607/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:40.475864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.589647 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8b3c88-59cf-436f-96ad-48d023c32d4e None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.626757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.627387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.627641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.627919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.628546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:40.628848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:40.630238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:40.644571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 618/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:40.651206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.756270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:40.764262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9befaf-0177-45e8-9ad7-6f6a812de676 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.798283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.798896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.799279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.799645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.810056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:40.812456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:40.824594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:40.825758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:40.826851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:40.829236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 608/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:40.837316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:40.886830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:40.945001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59210f0d-8e5d-41a2-97a7-6ae0b7787770 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:40.980103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:40.980743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:40.980985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:40.981257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:40.982000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:40.982299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:40.983717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:41.000786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 619/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:41.006947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:41.127209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37bdbc-aa31-4743-b403-b0dfa19bfce1 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:41.166410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:41.167275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:41.167542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:41.167851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:41.178549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:41.180021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:41.194083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:41.196204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:41.197389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:41.197389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:41.197389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:41.197389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 609/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:41.201619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:41.312918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f818825a-e32c-4ecc-96f3-9d0a32964327 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:41.315293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:41.353021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:41.353695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:41.354014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:41.354411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:41.355189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:41.355522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:41.357568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:41.370278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 620/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:41.376087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:41.387106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:41.490358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cee8032-cee5-40be-8dcf-920020d84b52 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:41.531141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:41.531855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:41.532314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:41.532635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:41.542539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:41.544326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:41.559684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:41.560820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:41.561874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 610/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:41.566898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:41.673453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37d352-8336-4f31-8b35-b7a06ba35855 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:41.710449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:41.711258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:41.711659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:41.712044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:41.712888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:41.713285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:41.715061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6249f54-a6cf-4a5f-9d3b-100d9f21ba32 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:41.734942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 621/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:41.736323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:41.809660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:41.854080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2a4f72-1bfc-439c-aaf4-ee990b090208 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:41.890869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145fde12-fb39-4a32-a7bb-e4ed02ac3c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:41.891521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fde12-fb39-4a32-a7bb-e4ed02ac3c86 None admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:41.891798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fde12-fb39-4a32-a7bb-e4ed02ac3c86 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:41.892071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145fde12-fb39-4a32-a7bb-e4ed02ac3c86 None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/1f3be656bf6446f99a23910cd833d799/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:41.907508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fde12-fb39-4a32-a7bb-e4ed02ac3c86 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=1f3be656bf6446f99a23910cd833d799, role_id=11c7a4c8a08a4af9ac197a11fc3ab6d9)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:41.909223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145fde12-fb39-4a32-a7bb-e4ed02ac3c86 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:41.938201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 611/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:40:41 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/1f3be656bf6446f99a23910cd833d799/roles/11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:40:44.287888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34541a-9ee8-4223-8544-5f68c3dd1ed4 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:44.289073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34541a-9ee8-4223-8544-5f68c3dd1ed4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:44.289073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff34541a-9ee8-4223-8544-5f68c3dd1ed4 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:44.290359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 622/1233] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:44.295913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634420-3326-452d-b513-8788d0789b6a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:44.296366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634420-3326-452d-b513-8788d0789b6a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:44.296486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634420-3326-452d-b513-8788d0789b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:44.316017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-29634420-3326-452d-b513-8788d0789b6a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:44.317432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-29634420-3326-452d-b513-8788d0789b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:44.392390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 612/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 19:40:44.656236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34541a-9ee8-4223-8544-5f68c3dd1ed4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:44.681680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:44.762170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff34541a-9ee8-4223-8544-5f68c3dd1ed4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:44.797202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:44.797868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:44.798150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:44.798421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:44.808717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:44.813481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:44.814667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:44.815586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 623/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Tue Sep 13 19:40:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:44.821010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29634420-3326-452d-b513-8788d0789b6a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:44.870959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:44.928319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29634420-3326-452d-b513-8788d0789b6a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:44.964123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:44.964869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:44.965207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:44.965539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:44.966243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:44.966525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:44.967877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:44.976153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 613/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Tue Sep 13 19:40:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:44.982875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:45.072462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:45.089859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7482f20-d43f-4bc4-854f-e545f7148f5c None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:45.124117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:45.124748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:45.125018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:45.125290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:45.135882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:45.137441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:45.153142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:45.160326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:45.161454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:45.161454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:45.161454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:45.161454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 624/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:45.161775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:45.267601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189a7cf7-0ff9-48ee-a3d9-7ec56ffe81f2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:45.277666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:45.304647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:45.305329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:45.305514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:45.305815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:45.306454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:45.306798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:45.308154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:45.321348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 614/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:45.327625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:45.436043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc99c670-e616-43ec-9614-9c5998970e11 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:45.472139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:45.472813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:45.473092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:45.473313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:45.476030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:45.481747 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:45.483222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:45.495211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:45.496338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:45.497434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 625/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:45.502280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:45.607688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7649d0ca-83ab-4125-8517-21432ca6eac6 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:45.648009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:45.648828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:45.649244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:45.649692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:45.650867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:45.651175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:45.652716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:45.680075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 615/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:45.681529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:45.796158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793110b-ee1d-4fdd-8c07-5f076caa0b15 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:45.834615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e551f40-3843-417c-b222-217bafd080da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:45.835434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e551f40-3843-417c-b222-217bafd080da None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:45.835807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e551f40-3843-417c-b222-217bafd080da None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:45.836177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e551f40-3843-417c-b222-217bafd080da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:45.904251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:45.912429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e551f40-3843-417c-b222-217bafd080da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:45.914706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e551f40-3843-417c-b222-217bafd080da None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.019173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 626/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:40:46.025265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.056615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:46.134937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8bdb02-2952-4bc6-bf79-38ec204baaed None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.173368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.173943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.174251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.174517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.185611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.187073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:46.201763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:46.202933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:46.203997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:46.203997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:46.203997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:46.203997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 616/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:46.208763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e551f40-3843-417c-b222-217bafd080da None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.314232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:46.320496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e551f40-3843-417c-b222-217bafd080da None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.357526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.358216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.358433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.358706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.359379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:46.359686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.361084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.372770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 627/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:46.378510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.473128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:46.501203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10551372-514b-486e-9a5e-e502ba1e03bb None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.543297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.544069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.544408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.544684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.552172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.553665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:46.566186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:46.567364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:46.568629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 617/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:46.573720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.683392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f91169-dc4f-4cde-9fbb-da2a3c263356 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.722589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.723236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.723466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.723736 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.724419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:46.724746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.726275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8957d069-cc7c-493e-bd9f-6826714b5f17 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.739771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 628/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:46.746345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:46.874519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454bb7c4-a94e-472f-9c2d-6281c6e70ba5 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:46.910525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:46.911123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:46.911399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:46.911685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:46.912393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] RBAC: Adding query filter params (role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=1f3be656bf6446f99a23910cd833d799) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:46.912688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:46.914755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104eba0-4551-4d9b-8f92-9346b56bf349 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:46.933637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 618/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:46 2022] GET /identity/v3/role_assignments?user.id=1f3be656bf6446f99a23910cd833d799&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=11c7a4c8a08a4af9ac197a11fc3ab6d9 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:49.641529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8a88-ebbf-4bc1-a5b1-4dbde8dc5e49 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:49.642105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8a88-ebbf-4bc1-a5b1-4dbde8dc5e49 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:49.642105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d8a88-ebbf-4bc1-a5b1-4dbde8dc5e49 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:49.643417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 629/1247] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:49.649284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:49.649686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:49.649850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:49.656169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:49.669520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:49.670758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:49.746313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 619/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:50.040705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8a88-ebbf-4bc1-a5b1-4dbde8dc5e49 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:50.061120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:50.146227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d8a88-ebbf-4bc1-a5b1-4dbde8dc5e49 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:50.184647 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:50.185246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.185526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.185836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.196797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:50.202528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:50.203662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:50.206722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 630/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:50.209871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:50.322220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84173a39-89fa-44d6-b095-cb9dd882a5a7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:50.353498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:50.359870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:50.360538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.360901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.361198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.362374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:50.362779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:50.364452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:50.374783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 620/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:40:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:50.381808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:50.470299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:50.504555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd7de17-0cbd-4a79-835b-b80c22ac94d1 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:50.541970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:50.542544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.542778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.543038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.552979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:50.555068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:50.569553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:50.570767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:50.572339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:50.572339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:50.572339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:50.572339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 631/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:50.576651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:50.687453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f7b2d-cd90-422a-b343-c28d4edbbd5a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:50.727410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:50.727410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.727410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.727410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.728146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:50.728460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:50.732386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:50.747645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 621/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:50.753569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:50.864908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63635a49-b7c5-4b85-aa0e-0a3b3ef8dfd8 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:50.895789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:50.904737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:50.905489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:50.905807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:50.906085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:50.915428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:50.917143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:50.928556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:50.929802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:50.930945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 632/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 19:40:50.935716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:50.999912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:51.050866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2cf9c-4915-4285-9adb-93b8abf4652a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:51.089677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:51.090254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.090507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.090779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.091523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:51.091803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:51.093239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:51.117290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 622/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:51.118220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:51.224268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e630a8a7-2cd6-4533-833f-629ebbf19786 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:51.260063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:51.260683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.260960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.261230 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.320270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:51.333302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:51.335431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:51.438739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 633/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 19:40:51.445063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:51.554800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5caa26-04e6-4d48-ae21-4a74d34fb352 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:51.567922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:51.589338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:51.589994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.590268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.590532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.600664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:51.602118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:51.617482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:51.618697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:51.619639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:51.619639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:51.619639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:51.619639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 623/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:51.624774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:51.714160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:51.727350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed1b77-155c-4b9a-abcd-950da2f601d4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:51.761707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:51.762380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.762612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.762887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.763527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:51.763829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:51.765190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:51.776292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 634/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:51.781935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:51.883667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab7883-9d23-4bb3-8a89-cd34a379935c None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:51.915712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:51.916575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:51.916848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:51.917118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] PATH_INFO: `/v3/projects/service` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:51.923466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:51.924867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:51.936279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:51.937322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:51.938448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 624/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:51.943321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:52.047772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ec8636-954c-461d-b581-3f86afce0cf2 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:52.080700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:52.081299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:52.081579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:52.081882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:52.082569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:52.082874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:52.084288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955756d-fc11-4108-ba1b-4dd2c89f11a4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:52.096305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 635/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:52.102894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:52.127392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:52.198601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f33ee2-77e1-458b-895a-c2b9b293710d None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:52.230680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:52.231250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:52.231521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:52.231784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:52.232481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=1f3be656bf6446f99a23910cd833d799) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:52.232776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:52.234187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2e627-4105-4313-8277-9545de1cd328 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:52.239225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 625/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:40:52 2022] GET /identity/v3/role_assignments?user.id=1f3be656bf6446f99a23910cd833d799&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=40a9647e8ec745c999c2457d053557a7 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:54.361701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65736fa2-1c8c-40d1-85f0-dd0cca4fe315 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:54.362272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65736fa2-1c8c-40d1-85f0-dd0cca4fe315 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:54.362272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65736fa2-1c8c-40d1-85f0-dd0cca4fe315 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:54.363587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 636/1261] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:54.369210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:54.369644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:54.369777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:54.388090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:54.389262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:54.453095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 626/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:54.730946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65736fa2-1c8c-40d1-85f0-dd0cca4fe315 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:54.742312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:54.828720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65736fa2-1c8c-40d1-85f0-dd0cca4fe315 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:54.861283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:54.861928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:54.862201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:54.862464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:54.872871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:54.877520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:54.878653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:54.879921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 637/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:40:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:54.884599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:54.902399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:54.982136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffe367c-b4b5-40db-8910-641ad90305bd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.015730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.016321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.016595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.016860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.017510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] RBAC: Adding query filter params (name=admin) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:55.017901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.019244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.026938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 627/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:40:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:55.032797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:55.098358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:55.131483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c0f141-f20a-4522-8054-4d089d5a701a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.165079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.165694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.166250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.166524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.176229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.177654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:55.191816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:55.192769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:55.193746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:55.193746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:55.193746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:55.193746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 638/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:55.198563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:55.276556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:55.295900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4065c3a-8290-47f9-9a2d-2860290cb00a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.327673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.328284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.328563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.328836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.329490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:55.329823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.331203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.344800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 628/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:55.351149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:55.447705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d8d28-4375-4fb1-9a98-c1dab4189537 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.466764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:55.480788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.481217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.481495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.481843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.490330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.492028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:55.504958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:55.506609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:55.507991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 639/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:55.513068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:55.610621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35361794-456a-4c9a-887e-57d45863f2e9 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.642971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.643555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.643831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.644104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.644816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:55.645117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.646544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.655390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:55.663808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 629/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:55.669926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:55.767523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f191e59-ca09-4684-b3d1-017c550a7bb7 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.799800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.800404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.800680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.800949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.810913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.812322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:55.826248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:55.827164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:55.828069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:55.828069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:55.828069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:55.828069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 640/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:55.833018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:55.928757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0dc0bfa-1c8d-424e-b4d4-c6bd61536278 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:55.961058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:55.961644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:55.961948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:55.962218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:55.962844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:55.963142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:55.964474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:55.975228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 630/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:55.980804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:56.012205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:56.078234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6cbe83-4387-4d06-a038-4e5200137135 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:56.109678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:56.110310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:56.110588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:56.110857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:56.117712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:56.119151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:40:56.130028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:56.131089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:40:56.133005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 641/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:40:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:56.137297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:56.192884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:56.233208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c7b1c8-9a3f-4475-83cd-4c59ac8e26ae None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:56.264918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:56.265499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:56.265795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:56.266065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:56.266739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:56.267040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:56.268447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e597339-e153-4255-9230-0bb6584091ae None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:56.280571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 631/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:40:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:40:56.286513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:56.373674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:56.382483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e9f06-fddf-4e3e-9fb9-53dd5a289ee4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:56.414205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2083a-5474-47d2-bdb9-667ac46d3b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:56.414835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2083a-5474-47d2-bdb9-667ac46d3b32 None admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:56.415112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2083a-5474-47d2-bdb9-667ac46d3b32 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:56.415379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2083a-5474-47d2-bdb9-667ac46d3b32 None admin] PATH_INFO: `/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/1f3be656bf6446f99a23910cd833d799/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:56.417483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2083a-5474-47d2-bdb9-667ac46d3b32 None admin] RBAC: Authorizing `identity:create_grant(project_id=ffb76500ee954275b63e4afa84c5b184, user_id=1f3be656bf6446f99a23910cd833d799, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:56.418926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2083a-5474-47d2-bdb9-667ac46d3b32 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:56.432085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 642/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Tue Sep 13 19:40:56 2022] PUT /identity/v3/projects/ffb76500ee954275b63e4afa84c5b184/users/1f3be656bf6446f99a23910cd833d799/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:40:58.536777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796532ff-2137-490e-bb81-849a955e6c03 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:58.537327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796532ff-2137-490e-bb81-849a955e6c03 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:58.537417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796532ff-2137-490e-bb81-849a955e6c03 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:58.538858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 632/1274] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:40:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:40:58.544812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:58.545277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:58.545399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:58.563669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:40:58.564869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:40:58.628924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 643/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:40:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:40:58.895174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796532ff-2137-490e-bb81-849a955e6c03 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:58.995486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796532ff-2137-490e-bb81-849a955e6c03 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.012324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:59.028650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.029259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.029532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.029826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:59.039490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 19:40:59.044433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:59.045501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:59.046406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 633/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Tue Sep 13 19:40:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:59.051680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:59.128439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:59.147801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654c98da-97c3-4a58-a524-12a5ffd9c6c8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.180213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.180798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.181071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.181338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:59.182009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:59.182306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:59.183641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:59.191235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 644/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Tue Sep 13 19:40:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:59.197452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:59.293953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f099010-59f6-4fec-b5a7-ebd1f3ee2aa2 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.326452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.327063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.327339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.327606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:59.337165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:59.338603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:40:59.352980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:59.354238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:40:59.355522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:40:59.355522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:40:59.355522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:59.355522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 634/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:59.359738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:59.455429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73be05c-552f-4341-9095-53bffca77623 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.489518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.490825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.491134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.491470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:59.492104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:59.492477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:59.494084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:59.508980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 645/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:40:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:59.515392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:59.549578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:59.617579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1deee22-2ad9-4bda-8be5-2a92284554ce None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.649590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.650230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.650506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.650774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] PATH_INFO: `/v3/users/placement` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:59.657886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:59.659292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 19:40:59.670844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:40:59.671950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:40:59.673293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 635/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:40:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:40:59.677717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:59.681506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:59.774941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79d79d0-a5d4-4871-9fed-21510caed864 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.808239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.808838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.809117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.809390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:40:59.810145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:40:59.810451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:40:59.811873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:40:59.828218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 646/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Tue Sep 13 19:40:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:40:59.834424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:40:59.921910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:40:59.931486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9207332-4223-4cee-91d4-161b2a11da54 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:40:59.964821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:40:59.965404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:40:59.965681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:40:59.965977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.030715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:00.032073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:00.125012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:00.135314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 636/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Tue Sep 13 19:40:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 19:41:00.141114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:00.201283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:00.238464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7860585d-f8c9-4726-9587-9df43b5212b4 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:00.270530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:00.271136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.271994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.272263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.281809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:00.283149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 19:41:00.297110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:41:00.298273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 19:41:00.299352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 19:41:00.299352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 19:41:00.299352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:41:00.299352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 647/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:41:00.303824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:00.402625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c305e-4786-45d2-8d0f-878daa0ba8a8 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:00.434974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:00.435552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.435822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.436085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.436706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:00.436998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:00.438352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:00.449751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 637/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Tue Sep 13 19:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:00.455295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:00.554175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c24bde-b52d-45a6-bc8a-9810a901cd68 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:00.566954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:00.587149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:00.587759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.588035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.588300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] PATH_INFO: `/v3/projects/service` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.594639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:00.596040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 19:41:00.606780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:41:00.607692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 19:41:00.608801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 648/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Tue Sep 13 19:41:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:41:00.613649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:00.662260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:00.711292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae66c390-d7e3-4e61-bea7-4be8ab277a92 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:00.743717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:00.744301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.744578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.744849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.745532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:00.745858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:00.747263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d0fe99-93d2-4968-9bdc-8b1bdda02236 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:00.758663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 638/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Tue Sep 13 19:41:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:00.764539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:00.862644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4c464-47cc-4113-ab1e-78ee58c0f21a None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:00.894787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:00.895374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:00.895649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:00.895916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:00.896622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] RBAC: Adding query filter params (role.id=40a9647e8ec745c999c2457d053557a7, scope.project.id=ffb76500ee954275b63e4afa84c5b184, user.id=1f3be656bf6446f99a23910cd833d799) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:00.896926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:00.898375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb63e7a-8e1b-4bda-95a7-cfbc7e48fbd1 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:00.903825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 649/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Tue Sep 13 19:41:00 2022] GET /identity/v3/role_assignments?user.id=1f3be656bf6446f99a23910cd833d799&scope.project.id=ffb76500ee954275b63e4afa84c5b184&role.id=40a9647e8ec745c999c2457d053557a7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:03.032735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a8224-1938-4195-bba9-065961ccb8b3 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:03.033301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a8224-1938-4195-bba9-065961ccb8b3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:03.033301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4a8224-1938-4195-bba9-065961ccb8b3 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:03.034616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 639/1288] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:03.039708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:03.040131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:03.040236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:03.058165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:03.059359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:03.088168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:03.123792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 650/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:41:03.385508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a8224-1938-4195-bba9-065961ccb8b3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:03.483417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4a8224-1938-4195-bba9-065961ccb8b3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:03.496006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:03.521925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:03.522550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:03.522826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:03.523104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:03.523706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:03.525039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:41:03.533971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:41:03.535623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:03.536801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 640/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:41:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:41:03.541789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:03.638569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda514a5-3f9f-43a2-bed5-faa7cb6d1ec5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:03.670546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:03.671138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:03.671413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:03.671680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:03.672291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:03.672587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:03.673960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4779b56-23c2-4e92-b265-e916d6b65625 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:03.679041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 651/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 19:41:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:03.684807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:03.784570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cb45a-950d-463c-bfe0-d2cc7beb23e4 None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:03.817532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:03.818265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:03.818539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:03.818803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] PATH_INFO: `/v3/services` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:03.819416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] RBAC: Adding query filter params (type=placement) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:03.819711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:03.821043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591c7957-d47a-4ec3-aaf3-4561645f93f6 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:03.825274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 641/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 19:41:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:05.967655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c96b-239d-4caf-bd5a-8555c4cdb47c None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:05.968175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c96b-239d-4caf-bd5a-8555c4cdb47c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:05.968175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45c96b-239d-4caf-bd5a-8555c4cdb47c None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:05.969496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 652/1293] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:05.974705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fb779-f6de-4bfb-8cec-14dc1e59d83e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:05.975142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fb779-f6de-4bfb-8cec-14dc1e59d83e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:05.975238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3fb779-f6de-4bfb-8cec-14dc1e59d83e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:05.993676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6a3fb779-f6de-4bfb-8cec-14dc1e59d83e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:05.994870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6a3fb779-f6de-4bfb-8cec-14dc1e59d83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:06.026057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:06.061947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 642/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:41:06.325031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45c96b-239d-4caf-bd5a-8555c4cdb47c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:06.355291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:06.422892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45c96b-239d-4caf-bd5a-8555c4cdb47c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:06.455557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0359a-b046-4817-aad3-df11a9b85f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:06.456143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0359a-b046-4817-aad3-df11a9b85f82 None admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:06.456416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0359a-b046-4817-aad3-df11a9b85f82 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:06.456686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0359a-b046-4817-aad3-df11a9b85f82 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:06.457324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0359a-b046-4817-aad3-df11a9b85f82 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:06.458713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0359a-b046-4817-aad3-df11a9b85f82 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:06.472331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 653/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Tue Sep 13 19:41:06 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:41:08.603347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ffa3f-69c8-410d-876d-e174410cbfce None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:08.603883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ffa3f-69c8-410d-876d-e174410cbfce None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:08.603883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2ffa3f-69c8-410d-876d-e174410cbfce None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:08.605187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 643/1296] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:41:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:08.610291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:08.610710 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:08.610844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:08.628583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:08.629798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:08.705021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 654/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:41:08.974078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ffa3f-69c8-410d-876d-e174410cbfce None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:09.072057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2ffa3f-69c8-410d-876d-e174410cbfce None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:09.104510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:09.105119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:09.105390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:09.105653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] PATH_INFO: `/v3/services/placement` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:09.106254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:09.107590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:41:09.116045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:41:09.117296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:09.118521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 644/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:41:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:41:09.123095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:09.140951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:09.219454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c33f5c-bd00-4bb1-9730-589d8caa3c88 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:09.251036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:09.251699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:09.251975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:09.252242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:09.252850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:09.253144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:09.254511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:09.260060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 655/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 19:41:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:09.265997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:09.297022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:09.362328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d2893-6dcc-49ed-a1f8-5cf194b39acd None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:09.394570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:09.395153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:09.395425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:09.395715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:09.396397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] RBAC: Adding query filter params (interface=public, service_id=3f8a169ba5024ec9bccd7184ae6f3e19, region_id=RegionOne) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:09.396697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:09.398058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd96d9-06ae-4f8a-ad4a-a9fa4b167c74 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:09.402812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 645/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Tue Sep 13 19:41:09 2022] GET /identity/v3/endpoints?service_id=3f8a169ba5024ec9bccd7184ae6f3e19&interface=public®ion_id=RegionOne => generated 194 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:09.408301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:09.487080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:09.508875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef553c-b199-474c-8743-aab332b171b9 None admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:09.544846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610dc6a0-23ef-407e-8479-4ff53ddae1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:09.545444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dc6a0-23ef-407e-8479-4ff53ddae1af None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:09.545720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dc6a0-23ef-407e-8479-4ff53ddae1af None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:09.546029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610dc6a0-23ef-407e-8479-4ff53ddae1af None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:09.546626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dc6a0-23ef-407e-8479-4ff53ddae1af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:09.547983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610dc6a0-23ef-407e-8479-4ff53ddae1af None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:09.555075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 656/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Tue Sep 13 19:41:09 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:41:11.657008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731f869-4877-432c-ad6f-48b2255db5a8 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:11.657571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731f869-4877-432c-ad6f-48b2255db5a8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:11.657571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731f869-4877-432c-ad6f-48b2255db5a8 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:11.658896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 646/1302] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:41:11.663961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:11.664357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:11.664511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:11.682940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:41:11.684120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:41:11.747640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 657/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:41:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 19:41:12.009881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731f869-4877-432c-ad6f-48b2255db5a8 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:12.100454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:12.107015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731f869-4877-432c-ad6f-48b2255db5a8 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:12.140303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:12.140904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:12.141174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:12.141435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] PATH_INFO: `/v3/services/placement` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:12.142027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:12.143355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 19:41:12.151735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 19:41:12.152921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 19:41:12.154161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 647/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Tue Sep 13 19:41:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 19:41:12.158771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:12.232288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:12.254880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d0422-323d-4ab3-a146-b1fc55002cee None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:12.287331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:12.287865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:12.288141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:12.288412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] PATH_INFO: `/v3/services` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:12.289031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] RBAC: Adding query filter params (name=placement) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:41:12.289331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:12.290983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ff6c5-acf1-4ed7-9b29-a5c110b6a180 None admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:12.295840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 658/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Tue Sep 13 19:41:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:41:12.301355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:41:12.396292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc5b67-f659-4ed1-853e-4bc14e77ce4a None admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:41:12.428709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb31456-ee5a-4263-84a2-b9dc933039c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:41:12.429275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb31456-ee5a-4263-84a2-b9dc933039c8 None admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:41:12.429546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb31456-ee5a-4263-84a2-b9dc933039c8 None admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:41:12.429839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb31456-ee5a-4263-84a2-b9dc933039c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:41:12.430481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb31456-ee5a-4263-84a2-b9dc933039c8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:41:12.431801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb31456-ee5a-4263-84a2-b9dc933039c8 None admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:41:12.450118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 648/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:41:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:41:22.293481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=30 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:41:22.446459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:16.611797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14d58-2059-4783-9d29-b493f43b54c4 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.612291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14d58-2059-4783-9d29-b493f43b54c4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.612538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf14d58-2059-4783-9d29-b493f43b54c4 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.618791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 659/1307] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:16.620248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ae9a5-e602-43ab-b4ad-1d529e5492ce None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:16.620720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ae9a5-e602-43ab-b4ad-1d529e5492ce None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:16.620824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7ae9a5-e602-43ab-b4ad-1d529e5492ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:16.733159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2f7ae9a5-e602-43ab-b4ad-1d529e5492ce None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:16.735126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2f7ae9a5-e602-43ab-b4ad-1d529e5492ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:16.877827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 649/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:17.175782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0a62-d1c1-4d07-a442-615daea16875 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:17.176062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0a62-d1c1-4d07-a442-615daea16875 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:17.176512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27db0a62-d1c1-4d07-a442-615daea16875 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:17.178521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 660/1309] 10.222.0.15 () {54 vars in 965 bytes} [Tue Sep 13 19:42:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:17.185243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab982a7-3daf-4c80-8052-509e66c79797 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:17.185565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab982a7-3daf-4c80-8052-509e66c79797 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:17.185831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab982a7-3daf-4c80-8052-509e66c79797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:17.285057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8ab982a7-3daf-4c80-8052-509e66c79797 None None] MFA Rules not processed for user `36fab0873915497cada62e8ea73bd4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:17.286340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8ab982a7-3daf-4c80-8052-509e66c79797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:17.312928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:17.380245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 650/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Tue Sep 13 19:42:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:17.389447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0a62-d1c1-4d07-a442-615daea16875 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:17.391937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-27db0a62-d1c1-4d07-a442-615daea16875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:17.509843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27db0a62-d1c1-4d07-a442-615daea16875 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:17.591914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:17.592547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:17.592940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:17.593279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:17.594981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:17.659364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:17.735301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:17.736900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186744c7-9ee6-4a54-b06f-cfcce86dcc89 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:17.761419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 661/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:20.216961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953a67-a871-4b60-b947-a87588aba975 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:20.217501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953a67-a871-4b60-b947-a87588aba975 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:20.217501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a953a67-a871-4b60-b947-a87588aba975 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:20.218899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 651/1312] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:20.223985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5dd09-3ba9-435e-bd04-a84ad4b82874 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:20.224356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5dd09-3ba9-435e-bd04-a84ad4b82874 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:20.224528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e5dd09-3ba9-435e-bd04-a84ad4b82874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:20.293541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-a4e5dd09-3ba9-435e-bd04-a84ad4b82874 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:20.294774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a4e5dd09-3ba9-435e-bd04-a84ad4b82874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:20.387577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 662/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:20.667692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a953a67-a871-4b60-b947-a87588aba975 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:20.669273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9a953a67-a871-4b60-b947-a87588aba975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:20.771434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a953a67-a871-4b60-b947-a87588aba975 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:20.853272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:20.855756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:20.856414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:20.856746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:20.857053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:20.859807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:20.965465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:20.966837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb1088c-b74a-4c61-a234-ad92659f3e68 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:20.985324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 652/1314] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:23.437146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dbdb2-4d66-4425-8705-f6f2dc760cf1 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:23.437684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dbdb2-4d66-4425-8705-f6f2dc760cf1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:23.437684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dbdb2-4d66-4425-8705-f6f2dc760cf1 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:23.439041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 663/1315] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:23.444399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca18189-7998-4fdf-be41-bec15feaa103 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:23.444868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca18189-7998-4fdf-be41-bec15feaa103 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:23.444975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca18189-7998-4fdf-be41-bec15feaa103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:23.512392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0ca18189-7998-4fdf-be41-bec15feaa103 None None] MFA Rules not processed for user `d7a3f9e9eac64a11840e9edae8f10588`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:23.513590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0ca18189-7998-4fdf-be41-bec15feaa103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:23.565628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:23.610295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 653/1316] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:23.893802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dbdb2-4d66-4425-8705-f6f2dc760cf1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:23.895332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-343dbdb2-4d66-4425-8705-f6f2dc760cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:23.905537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:23.995449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dbdb2-4d66-4425-8705-f6f2dc760cf1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:24.065918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:24.066507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:24.066781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:24.067046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:24.068604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:24.130373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:24.174723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:24.176149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9403642-c9b2-41af-b879-f25db7931d1e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:24.194244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 664/1317] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:26.847362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac9a77-dc08-4e4d-89c9-050894b438f8 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:26.847950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac9a77-dc08-4e4d-89c9-050894b438f8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:26.847950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac9a77-dc08-4e4d-89c9-050894b438f8 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:26.849343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 654/1318] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:26.854924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3310c7-c12e-4b25-8b19-bb3009202e64 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:26.855314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3310c7-c12e-4b25-8b19-bb3009202e64 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:26.855469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3310c7-c12e-4b25-8b19-bb3009202e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:26.919721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-0f3310c7-c12e-4b25-8b19-bb3009202e64 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:26.920959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0f3310c7-c12e-4b25-8b19-bb3009202e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:27.014786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 665/1319] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:26 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:27.295540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac9a77-dc08-4e4d-89c9-050894b438f8 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:27.297066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-00ac9a77-dc08-4e4d-89c9-050894b438f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:27.397627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac9a77-dc08-4e4d-89c9-050894b438f8 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:27.469505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:27.470286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:27.470557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:27.470822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:27.472370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:27.479025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:27.584296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:27.585643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c7163-b678-4730-a846-d130744ba2c4 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:27.603067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 655/1320] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:30.385087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887f5c7-5d0f-4c98-bebd-a714acd3082e None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:30.385612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887f5c7-5d0f-4c98-bebd-a714acd3082e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:30.385612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0887f5c7-5d0f-4c98-bebd-a714acd3082e None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:30.386888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 666/1321] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:30.392179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd726-45c5-4a9b-8d42-c87b6d019fe8 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:30.392602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd726-45c5-4a9b-8d42-c87b6d019fe8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:30.392780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85edd726-45c5-4a9b-8d42-c87b6d019fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:30.463661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-85edd726-45c5-4a9b-8d42-c87b6d019fe8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:30.464894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-85edd726-45c5-4a9b-8d42-c87b6d019fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:30.528180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:30.571066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 656/1322] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:30.853121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887f5c7-5d0f-4c98-bebd-a714acd3082e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:30.854757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0887f5c7-5d0f-4c98-bebd-a714acd3082e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:30.876865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:30.960703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0887f5c7-5d0f-4c98-bebd-a714acd3082e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:31.033083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:31.033685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:31.033994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:31.034269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:31.035873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:31.107654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:31.145756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:31.147191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca358cee-1b9a-46e5-9bfa-1e078fa643db service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:31.166541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 667/1323] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:33.399875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3daa65-5c3b-4e32-aed4-6a5a09cd914d None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:33.400405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3daa65-5c3b-4e32-aed4-6a5a09cd914d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:33.400405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3daa65-5c3b-4e32-aed4-6a5a09cd914d None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:33.401761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 657/1324] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:33.407189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c0dc5-3b43-4de1-a852-8ac3815f4d77 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:33.407641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c0dc5-3b43-4de1-a852-8ac3815f4d77 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:33.407987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2c0dc5-3b43-4de1-a852-8ac3815f4d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:33.473018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ba2c0dc5-3b43-4de1-a852-8ac3815f4d77 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:33.474247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ba2c0dc5-3b43-4de1-a852-8ac3815f4d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:33.580779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 668/1325] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:33.869366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3daa65-5c3b-4e32-aed4-6a5a09cd914d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:33.870985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ed3daa65-5c3b-4e32-aed4-6a5a09cd914d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:33.976322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3daa65-5c3b-4e32-aed4-6a5a09cd914d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:34.052269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:34.052869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:34.053145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:34.053415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:34.055069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:34.073452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:34.167572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:34.169006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b994efb2-b8db-4a15-b04a-db0ab49cee27 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:34.187893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 658/1326] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:36.401969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41062-0755-40a1-9013-8146ec13b129 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:36.402431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41062-0755-40a1-9013-8146ec13b129 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:36.402539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b41062-0755-40a1-9013-8146ec13b129 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:36.403874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 669/1327] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:36.409427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2567-4237-4051-aa47-613b33317aef None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:36.409895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2567-4237-4051-aa47-613b33317aef None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:36.410000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15f2567-4237-4051-aa47-613b33317aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:36.484549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b15f2567-4237-4051-aa47-613b33317aef None None] MFA Rules not processed for user `d7a3f9e9eac64a11840e9edae8f10588`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:36.486334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b15f2567-4237-4051-aa47-613b33317aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:36.571475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:36.603395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 659/1328] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:36 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:36.887291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41062-0755-40a1-9013-8146ec13b129 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:36.888886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-77b41062-0755-40a1-9013-8146ec13b129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:36.922558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:36.998109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b41062-0755-40a1-9013-8146ec13b129 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:37.070608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:37.071213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:37.071492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:37.071761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:37.073342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:37.151816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:37.184239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:37.185750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5aa707-7e38-423b-bd59-e013a05b867c service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:37.204355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 670/1329] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:39.574300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8c33-1fa0-465b-b960-776e1b7619bf None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:39.574868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8c33-1fa0-465b-b960-776e1b7619bf None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:39.574868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8c33-1fa0-465b-b960-776e1b7619bf None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:39.576256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 660/1330] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:39.581414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f3043-ec4e-469c-becb-8373d1ab01ba None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:39.581871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f3043-ec4e-469c-becb-8373d1ab01ba None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:39.581998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63f3043-ec4e-469c-becb-8373d1ab01ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:39.646067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-a63f3043-ec4e-469c-becb-8373d1ab01ba None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:39.647266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a63f3043-ec4e-469c-becb-8373d1ab01ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:39.741220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 671/1331] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:40.050420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a8c33-1fa0-465b-b960-776e1b7619bf None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:40.051941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-396a8c33-1fa0-465b-b960-776e1b7619bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:40.154958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396a8c33-1fa0-465b-b960-776e1b7619bf None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:40.234755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:40.235344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:40.235614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:40.235879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:40.237455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:40.264703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:40.346112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:40.347442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ffe829-e407-4c1b-8818-23b9fb04b5f7 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:40.364553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 661/1332] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:42.575543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb673d-efab-444d-b1a3-cfa0799601af None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:42.576103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb673d-efab-444d-b1a3-cfa0799601af None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:42.576103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb673d-efab-444d-b1a3-cfa0799601af None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:42.577462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 672/1333] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:42.582644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815b545-a444-4ba8-8341-6634ec43bc90 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:42.583025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815b545-a444-4ba8-8341-6634ec43bc90 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:42.583207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815b545-a444-4ba8-8341-6634ec43bc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:42.649021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c815b545-a444-4ba8-8341-6634ec43bc90 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:42.650488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c815b545-a444-4ba8-8341-6634ec43bc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:42.724897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:42.747428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 662/1334] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:43.035853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb673d-efab-444d-b1a3-cfa0799601af None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:43.037658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fffb673d-efab-444d-b1a3-cfa0799601af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:43.077409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:43.138794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb673d-efab-444d-b1a3-cfa0799601af None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:43.214176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:43.214783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:43.215062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:43.215335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:43.217578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:43.300185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:43.326110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:43.327545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51024f56-8f0e-4bc9-8548-4d2120b2f461 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:43.345759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 673/1335] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:45.812438 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e2054-1eeb-4d22-8605-e7e55901f808 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:45.812950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e2054-1eeb-4d22-8605-e7e55901f808 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:45.812950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5e2054-1eeb-4d22-8605-e7e55901f808 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:45.814354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 663/1336] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:45.819356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46080cc-e022-4e29-b61d-673861253a3e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:45.819762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46080cc-e022-4e29-b61d-673861253a3e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:45.819898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46080cc-e022-4e29-b61d-673861253a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:45.884237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f46080cc-e022-4e29-b61d-673861253a3e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:45.885431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f46080cc-e022-4e29-b61d-673861253a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:45.977681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 674/1337] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:45 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:46.255769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e2054-1eeb-4d22-8605-e7e55901f808 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:46.257283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5f5e2054-1eeb-4d22-8605-e7e55901f808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:46.358073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5e2054-1eeb-4d22-8605-e7e55901f808 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:46.428772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:46.429351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:46.429621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:46.429917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:46.431463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:46.466203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:46.542871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:46.544285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0c468-e90d-4627-97e8-4742f43cdb17 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:46.562509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 664/1338] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:50.596397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463fd1-c16f-42ff-99c2-c0c11d25338f None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:50.596947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463fd1-c16f-42ff-99c2-c0c11d25338f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:50.596947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e463fd1-c16f-42ff-99c2-c0c11d25338f None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:50.598209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 675/1339] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:50.603428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8c782-bb96-4db0-b312-7b92adf5aaab None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:50.603885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8c782-bb96-4db0-b312-7b92adf5aaab None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:50.604117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb8c782-bb96-4db0-b312-7b92adf5aaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:50.670727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4fb8c782-bb96-4db0-b312-7b92adf5aaab None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:50.671842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4fb8c782-bb96-4db0-b312-7b92adf5aaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:50.758364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:50.770192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 665/1340] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:51.062073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463fd1-c16f-42ff-99c2-c0c11d25338f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:51.063701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5e463fd1-c16f-42ff-99c2-c0c11d25338f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:51.111889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:51.167209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e463fd1-c16f-42ff-99c2-c0c11d25338f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:51.247132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:51.248300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:51.248584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:51.248854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:51.250473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:51.343165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:51.359822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:51.361254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a2192-19bb-46ee-a49a-88d53225ca9e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:51.379942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 676/1341] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:54.183365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a276d-8b7f-40c9-8599-2385fb5e3318 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:54.183919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a276d-8b7f-40c9-8599-2385fb5e3318 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:54.183919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61a276d-8b7f-40c9-8599-2385fb5e3318 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:54.186015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 666/1342] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:54.190975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141af4c-678b-4079-9670-276f12d5b9ff None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:54.191370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141af4c-678b-4079-9670-276f12d5b9ff None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:54.191560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141af4c-678b-4079-9670-276f12d5b9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:54.256996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4141af4c-678b-4079-9670-276f12d5b9ff None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:54.258231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4141af4c-678b-4079-9670-276f12d5b9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:54.355652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 677/1343] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:54.639412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a276d-8b7f-40c9-8599-2385fb5e3318 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:54.640930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c61a276d-8b7f-40c9-8599-2385fb5e3318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:54.750842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61a276d-8b7f-40c9-8599-2385fb5e3318 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:54.827992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:54.828597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:54.828868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:54.829129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:54.830755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:54.876127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:54.946393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:54.947806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f8724-73d5-49d5-bc6a-eef2b8b50c04 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:54.966085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 667/1344] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:42:59.117789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfa822-e1ad-4f9a-b2c5-8a7d048a3e19 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:59.118330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfa822-e1ad-4f9a-b2c5-8a7d048a3e19 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:59.118330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bfa822-e1ad-4f9a-b2c5-8a7d048a3e19 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:59.119685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 678/1345] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:42:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:42:59.124789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840392-db11-44dc-acdf-a20323ee9767 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:59.125185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840392-db11-44dc-acdf-a20323ee9767 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:59.125334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3840392-db11-44dc-acdf-a20323ee9767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:59.200236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a3840392-db11-44dc-acdf-a20323ee9767 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:42:59.202025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a3840392-db11-44dc-acdf-a20323ee9767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:59.295897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:59.298521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 668/1346] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:42:59.586871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfa822-e1ad-4f9a-b2c5-8a7d048a3e19 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:42:59.588409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-67bfa822-e1ad-4f9a-b2c5-8a7d048a3e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:59.645654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:59.691019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bfa822-e1ad-4f9a-b2c5-8a7d048a3e19 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:42:59.762104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:42:59.762701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:42:59.762980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:42:59.763282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:42:59.764862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:42:59.869640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:42:59.874284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:42:59.876149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69325142-08ba-4e0f-b979-d03f9597c2d9 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:42:59.894363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 679/1347] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:42:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:04.759743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5ac26-a424-435b-964c-ebe3750f7080 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:04.760254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5ac26-a424-435b-964c-ebe3750f7080 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:04.760552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d5ac26-a424-435b-964c-ebe3750f7080 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:04.762395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 669/1348] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:43:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:04.767867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8355fc-2004-493b-a5e8-5368e39f0443 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:04.768916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8355fc-2004-493b-a5e8-5368e39f0443 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:04.769405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8355fc-2004-493b-a5e8-5368e39f0443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:04.867272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-df8355fc-2004-493b-a5e8-5368e39f0443 None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:04.868471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-df8355fc-2004-493b-a5e8-5368e39f0443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:05.009188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 680/1349] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:43:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:09.282015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc5a7-c5ba-4f09-9efc-eba2973e7c72 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:09.282567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc5a7-c5ba-4f09-9efc-eba2973e7c72 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:09.282567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffc5a7-c5ba-4f09-9efc-eba2973e7c72 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:09.284049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 670/1350] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:09.289161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa90a1-7030-455a-a2a1-9fc7ff063dc2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:09.289612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa90a1-7030-455a-a2a1-9fc7ff063dc2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:09.289711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa90a1-7030-455a-a2a1-9fc7ff063dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:09.344690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:09.369070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-85aa90a1-7030-455a-a2a1-9fc7ff063dc2 None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:09.370300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-85aa90a1-7030-455a-a2a1-9fc7ff063dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:09.479820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 681/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:43:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:10.929475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591322ce-ba70-4041-82d0-b54f2b16af53 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:10.930039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591322ce-ba70-4041-82d0-b54f2b16af53 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:10.930039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591322ce-ba70-4041-82d0-b54f2b16af53 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:10.931388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 671/1352] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:10.937050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0904458-5f84-4515-81b5-ab6234b38ac8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:10.937463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0904458-5f84-4515-81b5-ab6234b38ac8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:10.937742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0904458-5f84-4515-81b5-ab6234b38ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:11.020277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e0904458-5f84-4515-81b5-ab6234b38ac8 None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:11.022006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e0904458-5f84-4515-81b5-ab6234b38ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:11.064400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:11.135953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 682/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:43:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:14.557434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a031-ea35-483c-bb44-133c80ea2aa5 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:14.558072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a031-ea35-483c-bb44-133c80ea2aa5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:14.558072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87a031-ea35-483c-bb44-133c80ea2aa5 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:14.559354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 672/1354] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:14.565067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c79b6-f730-4ca6-8e01-dd35c1e5fc0c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:14.565535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c79b6-f730-4ca6-8e01-dd35c1e5fc0c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:14.565651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c79b6-f730-4ca6-8e01-dd35c1e5fc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:14.647264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-576c79b6-f730-4ca6-8e01-dd35c1e5fc0c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:14.648998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-576c79b6-f730-4ca6-8e01-dd35c1e5fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:14.755443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 683/1355] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:15.066251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e43bc5-7e33-4e05-8159-a35d22b9b756 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:15.066980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e43bc5-7e33-4e05-8159-a35d22b9b756 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:15.067493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e43bc5-7e33-4e05-8159-a35d22b9b756 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:15.069067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 673/1356] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 19:43:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:15.077378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8695dc-8160-46a9-a44a-1504139f3a92 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:15.079302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8695dc-8160-46a9-a44a-1504139f3a92 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:15.079896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8695dc-8160-46a9-a44a-1504139f3a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:15.099921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:15.167672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-0a8695dc-8160-46a9-a44a-1504139f3a92 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:15.168858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0a8695dc-8160-46a9-a44a-1504139f3a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:15.292258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 684/1357] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 19:43:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:15.300611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e43bc5-7e33-4e05-8159-a35d22b9b756 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:15.302445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-92e43bc5-7e33-4e05-8159-a35d22b9b756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:15.473166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e43bc5-7e33-4e05-8159-a35d22b9b756 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:15.556292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:15.598330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:15.598973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:15.599244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:15.599484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:15.601017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:15.706145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:15.707725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630e280-aa44-4b48-9035-073b14a20a9a service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:15.727251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 674/1358] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:18.161725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1e883-65c4-4f29-86be-9f47f4f672e5 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.162176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1e883-65c4-4f29-86be-9f47f4f672e5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.162269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1e883-65c4-4f29-86be-9f47f4f672e5 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.169380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f125e-db38-4f8d-b661-f6598d077a0e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.169872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f125e-db38-4f8d-b661-f6598d077a0e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.169872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f125e-db38-4f8d-b661-f6598d077a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.171118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 685/1359] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:18.239932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-268f125e-db38-4f8d-b661-f6598d077a0e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:18.241214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-268f125e-db38-4f8d-b661-f6598d077a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.279916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:18.349937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 675/1360] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:18.642829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1e883-65c4-4f29-86be-9f47f4f672e5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:18.644454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-73b1e883-65c4-4f29-86be-9f47f4f672e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:18.729459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:18.845411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1e883-65c4-4f29-86be-9f47f4f672e5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:18.978261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:18.978877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:18.979156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:18.979428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:18.981042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:19.005615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:19.078921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:19.080532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749ba22b-e38d-4e27-a68c-27bd5ac90ff6 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:19.099673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 686/1361] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:21.581300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec4fa-cf4d-487a-a8cc-b04966e91555 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:21.582007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec4fa-cf4d-487a-a8cc-b04966e91555 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:21.582007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dec4fa-cf4d-487a-a8cc-b04966e91555 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:21.583470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 676/1362] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:21.588967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d41d-036b-42eb-93d2-90c5127b4186 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:21.589370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d41d-036b-42eb-93d2-90c5127b4186 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:21.589543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a9d41d-036b-42eb-93d2-90c5127b4186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:21.660718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-38a9d41d-036b-42eb-93d2-90c5127b4186 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:21.661893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-38a9d41d-036b-42eb-93d2-90c5127b4186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:21.764645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:21.767019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 687/1363] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:22.066413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ced8e-b0c5-41ff-9b2c-9ffb8f66c711 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.066955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ced8e-b0c5-41ff-9b2c-9ffb8f66c711 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.067299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ced8e-b0c5-41ff-9b2c-9ffb8f66c711 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.068717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 677/1364] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 19:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:22.074189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d078366-83d0-42fd-a086-433f2591e84c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.074589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d078366-83d0-42fd-a086-433f2591e84c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.074953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d078366-83d0-42fd-a086-433f2591e84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.158780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8d078366-83d0-42fd-a086-433f2591e84c None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:22.160072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8d078366-83d0-42fd-a086-433f2591e84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:22.280298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 688/1365] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 19:43:22 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:22.287885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ced8e-b0c5-41ff-9b2c-9ffb8f66c711 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:22.289640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ee7ced8e-b0c5-41ff-9b2c-9ffb8f66c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:22.447621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ced8e-b0c5-41ff-9b2c-9ffb8f66c711 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:22.469995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:22.571875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:22.572501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.572769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.573037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.574766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:22.678980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:22.680675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf9592-ae53-446b-b2e7-57addc2e855a service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:22.695735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dbc9b-0991-4d90-9422-38fbfc45ad3a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.696257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dbc9b-0991-4d90-9422-38fbfc45ad3a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.696586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5dbc9b-0991-4d90-9422-38fbfc45ad3a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.698116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 689/1366] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:43:22.703865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091dbf1-1429-4715-a091-2c4f84c66c7c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:22.704190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091dbf1-1429-4715-a091-2c4f84c66c7c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:22.704407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091dbf1-1429-4715-a091-2c4f84c66c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:22.704895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 678/1367] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:22.782220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:22.790851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c091dbf1-1429-4715-a091-2c4f84c66c7c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:22.792048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c091dbf1-1429-4715-a091-2c4f84c66c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:22.921317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 690/1368] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:23.275424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a64458-0253-4402-aebd-2a7fcf12d552 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.275717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a64458-0253-4402-aebd-2a7fcf12d552 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.275997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a64458-0253-4402-aebd-2a7fcf12d552 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.277474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 679/1369] 10.222.0.15 () {54 vars in 964 bytes} [Tue Sep 13 19:43:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:23.283148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fafbc-973e-4d25-ab0f-b53363181462 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.283577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fafbc-973e-4d25-ab0f-b53363181462 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.283950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fafbc-973e-4d25-ab0f-b53363181462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.366416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-827fafbc-973e-4d25-ab0f-b53363181462 None None] MFA Rules not processed for user `0f8ea3cf81f944a2b02e59b3ff049f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:23.368234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-827fafbc-973e-4d25-ab0f-b53363181462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.455646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.464321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 691/1370] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Sep 13 19:43:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:23.472562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a64458-0253-4402-aebd-2a7fcf12d552 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:23.474231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-44a64458-0253-4402-aebd-2a7fcf12d552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.505424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.587217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a64458-0253-4402-aebd-2a7fcf12d552 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:23.676931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:23.677568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:23.677877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:23.678160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:23.679921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:23.790389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:23.840307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:23.842961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209284c6-f2c9-407d-bfa5-084407b5fafd service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:23.867377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 680/1371] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:24.951078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819217-113a-4df3-b566-925cd208b167 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.951634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819217-113a-4df3-b566-925cd208b167 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.951634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f819217-113a-4df3-b566-925cd208b167 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:24.952926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 692/1372] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:24.958358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1466493-db1a-451d-908f-05e2dd107c5d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:24.958807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1466493-db1a-451d-908f-05e2dd107c5d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:24.959108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1466493-db1a-451d-908f-05e2dd107c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:25.028037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a1466493-db1a-451d-908f-05e2dd107c5d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:25.029811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a1466493-db1a-451d-908f-05e2dd107c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:25.128687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 681/1373] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:25.425581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819217-113a-4df3-b566-925cd208b167 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:25.427162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8f819217-113a-4df3-b566-925cd208b167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:25.578441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f819217-113a-4df3-b566-925cd208b167 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:25.645753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:25.693862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:25.694389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:25.694665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:25.694941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:25.696531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:25.791576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:25.793173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ee0bb-fe39-4c1e-855b-29ca8c1edc98 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:25.811891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 693/1374] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:28.054935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb003-3356-4790-94fa-c13c9ba6b5d1 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.055516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb003-3356-4790-94fa-c13c9ba6b5d1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.055516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eeb003-3356-4790-94fa-c13c9ba6b5d1 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.056858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 682/1375] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:28.062879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b972b-4f1f-4fdf-8c25-2292c177784a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.063440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b972b-4f1f-4fdf-8c25-2292c177784a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.063440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612b972b-4f1f-4fdf-8c25-2292c177784a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.134826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-612b972b-4f1f-4fdf-8c25-2292c177784a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:28.136046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-612b972b-4f1f-4fdf-8c25-2292c177784a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:28.169510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:28.242079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 694/1376] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:28.532089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb003-3356-4790-94fa-c13c9ba6b5d1 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:28.533702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-38eeb003-3356-4790-94fa-c13c9ba6b5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:28.573765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:28.688383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eeb003-3356-4790-94fa-c13c9ba6b5d1 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:28.805882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:28.812966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:28.813560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:28.813861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:28.814127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:28.815715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:28.920174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:28.922509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbba9a3-7b23-44a6-bf8d-8ad5bfb31185 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:28.941631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 683/1377] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:31.156725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e106e45-8094-42a0-acab-c28e1ec1e9d5 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:31.156725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e106e45-8094-42a0-acab-c28e1ec1e9d5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:31.157506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e106e45-8094-42a0-acab-c28e1ec1e9d5 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:31.158767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 695/1378] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:31.164716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686ccd6-5cec-4ea6-ad9a-8b155e833888 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:31.164716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686ccd6-5cec-4ea6-ad9a-8b155e833888 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:31.165483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686ccd6-5cec-4ea6-ad9a-8b155e833888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:31.255242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2686ccd6-5cec-4ea6-ad9a-8b155e833888 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:31.255836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2686ccd6-5cec-4ea6-ad9a-8b155e833888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:31.331395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:31.373034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 684/1379] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:31.744005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e106e45-8094-42a0-acab-c28e1ec1e9d5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:31.745518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1e106e45-8094-42a0-acab-c28e1ec1e9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:31.892222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e106e45-8094-42a0-acab-c28e1ec1e9d5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:31.905759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:32.006666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:32.007268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:32.007541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:32.007807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:32.009406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:32.101116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:32.102730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a47ff3b-e079-433d-b283-d5f3f6db5d81 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:32.121369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 696/1380] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:34.387606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc104c-a8e0-4168-85ff-89207cd5a4c5 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:34.388508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc104c-a8e0-4168-85ff-89207cd5a4c5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:34.388508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc104c-a8e0-4168-85ff-89207cd5a4c5 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:34.389374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 685/1381] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:34.396571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2976-903b-443b-88c8-57dbc9313670 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:34.397266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2976-903b-443b-88c8-57dbc9313670 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:34.397745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f2976-903b-443b-88c8-57dbc9313670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:34.425305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:34.479736 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4c6f2976-903b-443b-88c8-57dbc9313670 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:34.481009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4c6f2976-903b-443b-88c8-57dbc9313670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:34.584033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 697/1382] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:34.954551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc104c-a8e0-4168-85ff-89207cd5a4c5 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:34.956539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2cfc104c-a8e0-4168-85ff-89207cd5a4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:35.116185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc104c-a8e0-4168-85ff-89207cd5a4c5 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:35.173064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:35.242164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:35.242769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:35.243079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:35.243313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:35.244928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:35.346338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:35.348488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cbcd44-9187-4635-9887-a81a220e124c service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:35.368173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 686/1383] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:37.606092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b37a3-c610-40c4-a73c-e64f1bf3f572 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:37.608571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b37a3-c610-40c4-a73c-e64f1bf3f572 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:37.608571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b37a3-c610-40c4-a73c-e64f1bf3f572 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:37.608770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 698/1384] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:37.613983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348f23b-47f4-4960-b78d-88ef01fd51ca None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:37.614449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348f23b-47f4-4960-b78d-88ef01fd51ca None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:37.614560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4348f23b-47f4-4960-b78d-88ef01fd51ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:37.680518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4348f23b-47f4-4960-b78d-88ef01fd51ca None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:37.681718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4348f23b-47f4-4960-b78d-88ef01fd51ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:37.689899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:37.782300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 687/1385] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:38.068535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b37a3-c610-40c4-a73c-e64f1bf3f572 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:38.070062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0c2b37a3-c610-40c4-a73c-e64f1bf3f572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:38.163420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:38.218274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b37a3-c610-40c4-a73c-e64f1bf3f572 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:38.328526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:38.329092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:38.329367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:38.329631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:38.331289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:38.392272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:38.423852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:38.425470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d8cd4e-5894-44f1-a649-ffc9b0e676b7 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:38.448938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 699/1386] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:39.066735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a404a34-0d76-4bdb-8034-f495b67ea1e1 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.067297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a404a34-0d76-4bdb-8034-f495b67ea1e1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.067297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a404a34-0d76-4bdb-8034-f495b67ea1e1 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.068678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 688/1387] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:39.074191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c10bd-1784-4dea-8835-b3dd7fd1a38d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.074645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c10bd-1784-4dea-8835-b3dd7fd1a38d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.074745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c10bd-1784-4dea-8835-b3dd7fd1a38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.147615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-582c10bd-1784-4dea-8835-b3dd7fd1a38d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:39.148737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-582c10bd-1784-4dea-8835-b3dd7fd1a38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:39.246568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 700/1388] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:39.548601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9e08-fa55-4abb-aa85-8c7cd63f76d7 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.548967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9e08-fa55-4abb-aa85-8c7cd63f76d7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.549381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2a9e08-fa55-4abb-aa85-8c7cd63f76d7 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.550814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 689/1389] 10.222.0.15 () {54 vars in 966 bytes} [Tue Sep 13 19:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:39.555963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137336-f556-4127-bc97-4937080c0c7b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.556271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137336-f556-4127-bc97-4937080c0c7b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.556544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137336-f556-4127-bc97-4937080c0c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.602013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:39.633367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c8137336-f556-4127-bc97-4937080c0c7b None None] MFA Rules not processed for user `c21dea41dba24bf88466a7c3f00b8c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:39.634601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c8137336-f556-4127-bc97-4937080c0c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:39.712493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 701/1390] 10.222.0.15 () {58 vars in 1063 bytes} [Tue Sep 13 19:43:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:43:39.721553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a9e08-fa55-4abb-aa85-8c7cd63f76d7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:39.723091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ea2a9e08-fa55-4abb-aa85-8c7cd63f76d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:39.825719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2a9e08-fa55-4abb-aa85-8c7cd63f76d7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:39.863933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:39.900588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:39.901191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:39.901465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:39.901751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:39.903340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.010539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:40.011952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6257e64-aed9-4f1e-813b-bf5766d6286c service glance] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:40.030213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 690/1391] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:40.695585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbe0c5-2799-43f2-9ab4-8b617a8dec28 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.696119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbe0c5-2799-43f2-9ab4-8b617a8dec28 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.696119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbe0c5-2799-43f2-9ab4-8b617a8dec28 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.697505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 702/1392] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:40.702869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266091c-e382-4749-b20c-bdcd2f731934 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.703328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266091c-e382-4749-b20c-bdcd2f731934 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.703435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266091c-e382-4749-b20c-bdcd2f731934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.780956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada71eb9-d9d4-40bf-a28a-2afa927d04eb None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.781311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada71eb9-d9d4-40bf-a28a-2afa927d04eb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.782137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada71eb9-d9d4-40bf-a28a-2afa927d04eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.784208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1266091c-e382-4749-b20c-bdcd2f731934 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:40.785402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1266091c-e382-4749-b20c-bdcd2f731934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.804561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.856054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ada71eb9-d9d4-40bf-a28a-2afa927d04eb None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:40.857272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ada71eb9-d9d4-40bf-a28a-2afa927d04eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:40.890119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 691/1393] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:40.938099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:40.945744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 703/1394] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:43:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:40.991400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d041d-798d-49e8-a022-ef5d31182cf1 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:40.991683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d041d-798d-49e8-a022-ef5d31182cf1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:40.992097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1d041d-798d-49e8-a022-ef5d31182cf1 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:40.993461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 692/1395] 10.222.0.15 () {54 vars in 943 bytes} [Tue Sep 13 19:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:41.000010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.000295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.000570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.076648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] MFA Rules not processed for user `38d08c9aa2604871bec462424708b85b`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:41.077869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:41.184678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 704/1396] 10.222.0.15 () {58 vars in 1040 bytes} [Tue Sep 13 19:43:40 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:41.184857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d041d-798d-49e8-a022-ef5d31182cf1 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.187009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0a1d041d-798d-49e8-a022-ef5d31182cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:41.194479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:41.204858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69864f-2dfb-41ab-94db-643ef10c5e07 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.254631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.316664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.317270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.317546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.317852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.319530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:41.352993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1d041d-798d-49e8-a022-ef5d31182cf1 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:41.356215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:41.411700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.413652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf5920-743f-4d3a-a360-c25c50cc91ed service swift] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.422630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 705/1397] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 19:43:41.485690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:41.486508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:41.486914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:41.487222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:41.489102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:41.594942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:41.596537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33071942-eb7e-486b-b5b3-63098e021e01 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:41.617102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 693/1398] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:43:43.830125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028e2bf-5046-4f39-99cc-1ac5c8df7775 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:43.833169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028e2bf-5046-4f39-99cc-1ac5c8df7775 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:43.833814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8028e2bf-5046-4f39-99cc-1ac5c8df7775 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:43.835408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 706/1399] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:43.845488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805864f-5c79-4400-9dfd-62781588446f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:43.846057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805864f-5c79-4400-9dfd-62781588446f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:43.846057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805864f-5c79-4400-9dfd-62781588446f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:43.856668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:43.918085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8805864f-5c79-4400-9dfd-62781588446f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:43.919646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8805864f-5c79-4400-9dfd-62781588446f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:44.024882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 694/1400] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:44.315949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028e2bf-5046-4f39-99cc-1ac5c8df7775 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:44.317469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8028e2bf-5046-4f39-99cc-1ac5c8df7775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:44.404058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:44.476236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8028e2bf-5046-4f39-99cc-1ac5c8df7775 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:44.606093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:44.606697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:44.606976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:44.607759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:44.609371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:44.659477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:44.713204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:44.715080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bc1514-0f66-4011-8756-074d78be93de service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:44.734623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 707/1401] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:47.003460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f327d-23b3-48e0-8e7c-174978d20193 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.004025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f327d-23b3-48e0-8e7c-174978d20193 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.004025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f327d-23b3-48e0-8e7c-174978d20193 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.005521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 695/1402] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:47.010818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6a22d-394c-4a7a-884e-af4f5a4e5675 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.011582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6a22d-394c-4a7a-884e-af4f5a4e5675 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.011861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b6a22d-394c-4a7a-884e-af4f5a4e5675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.088527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f3b6a22d-394c-4a7a-884e-af4f5a4e5675 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:47.089781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f3b6a22d-394c-4a7a-884e-af4f5a4e5675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.212273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 708/1403] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:47.509558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f327d-23b3-48e0-8e7c-174978d20193 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:47.511244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-111f327d-23b3-48e0-8e7c-174978d20193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.613236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.688051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f327d-23b3-48e0-8e7c-174978d20193 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:47.846862 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:47.847464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:47.847736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:47.848002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:47.849636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:47.899821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:47.945943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:47.947846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58260c4-4a6d-43c7-a382-db1b45a7f10c service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:47.967542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 696/1404] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:43:50.175962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297cd9-7417-4eab-bd6c-bbd4041af804 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.176513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297cd9-7417-4eab-bd6c-bbd4041af804 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.176513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297cd9-7417-4eab-bd6c-bbd4041af804 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.177980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 709/1405] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:43:50.185147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adae68-c2d0-4d67-ad04-b32bb91d2d8b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.186170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adae68-c2d0-4d67-ad04-b32bb91d2d8b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.186306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91adae68-c2d0-4d67-ad04-b32bb91d2d8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.259458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-91adae68-c2d0-4d67-ad04-b32bb91d2d8b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:43:50.260674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-91adae68-c2d0-4d67-ad04-b32bb91d2d8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:50.366309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 697/1406] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:43:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:43:50.661699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297cd9-7417-4eab-bd6c-bbd4041af804 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:43:50.663389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bd297cd9-7417-4eab-bd6c-bbd4041af804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:50.689310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:50.838638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297cd9-7417-4eab-bd6c-bbd4041af804 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:43:50.939729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:43:50.966695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:43:50.967371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:43:50.967773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:43:50.968053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:43:50.969699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:43:51.074438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:43:51.077011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e6136-cdac-45b2-bad3-2a0de15f5ff5 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:43:51.095981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 710/1407] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:43:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:00.364417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:01.093723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=54 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:04.316642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbc0a1-f75f-446a-a2a2-e8b51799bcbe None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.317175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbc0a1-f75f-446a-a2a2-e8b51799bcbe None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.317175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbc0a1-f75f-446a-a2a2-e8b51799bcbe None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.318534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 698/1408] 10.222.0.15 () {54 vars in 918 bytes} [Tue Sep 13 19:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:04.323642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf935075-953d-4ed3-9ebe-90d3b512a4b7 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.324041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf935075-953d-4ed3-9ebe-90d3b512a4b7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.324218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf935075-953d-4ed3-9ebe-90d3b512a4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.397926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cf935075-953d-4ed3-9ebe-90d3b512a4b7 None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:04.399188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cf935075-953d-4ed3-9ebe-90d3b512a4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:04.503431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 711/1409] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Sep 13 19:44:04 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:04.517366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57fc7d-82c8-49a5-89a1-cfcce153d67c None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.517644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57fc7d-82c8-49a5-89a1-cfcce153d67c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.517942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b57fc7d-82c8-49a5-89a1-cfcce153d67c None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.519246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 699/1410] 10.222.0.15 () {54 vars in 918 bytes} [Tue Sep 13 19:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:04.523883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e13a3e-388a-485c-9d62-6a79a7f21422 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:04.524161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e13a3e-388a-485c-9d62-6a79a7f21422 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:04.524430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e13a3e-388a-485c-9d62-6a79a7f21422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:04.606049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:04.611146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-52e13a3e-388a-485c-9d62-6a79a7f21422 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:04.613139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-52e13a3e-388a-485c-9d62-6a79a7f21422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:04.735097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 712/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Tue Sep 13 19:44:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:44:07.297514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f51b0-62e5-4bed-8333-f2feae225753 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.298124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f51b0-62e5-4bed-8333-f2feae225753 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.298124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89f51b0-62e5-4bed-8333-f2feae225753 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.299417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 700/1412] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:07.304791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e41b8-644e-4663-8fce-88f06e0f8f72 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:07.305220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e41b8-644e-4663-8fce-88f06e0f8f72 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:07.305369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6e41b8-644e-4663-8fce-88f06e0f8f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:07.369832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-fd6e41b8-644e-4663-8fce-88f06e0f8f72 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:07.371054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fd6e41b8-644e-4663-8fce-88f06e0f8f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:07.424320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:07.465387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 713/1413] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:07.754734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f51b0-62e5-4bed-8333-f2feae225753 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:07.756287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b89f51b0-62e5-4bed-8333-f2feae225753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:07.898835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89f51b0-62e5-4bed-8333-f2feae225753 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:07.971483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:08.009795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:08.010336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:08.010603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:08.010868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:08.012437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:08.099233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:08.100792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d638a-9f39-4550-b3c4-c3d6b6e37dc8 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:08.118064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 701/1414] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:09.267448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3e9e-3ef2-4768-9ad7-dd915aa6bb8d None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.268014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3e9e-3ef2-4768-9ad7-dd915aa6bb8d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.268014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fc3e9e-3ef2-4768-9ad7-dd915aa6bb8d None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.269300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 714/1415] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:09.275601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a17a4-fb7f-4eec-8be8-5b007f7a1853 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.276044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a17a4-fb7f-4eec-8be8-5b007f7a1853 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.276157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a17a4-fb7f-4eec-8be8-5b007f7a1853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.351487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-230a17a4-fb7f-4eec-8be8-5b007f7a1853 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:09.352672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-230a17a4-fb7f-4eec-8be8-5b007f7a1853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.378859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:09.449167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 702/1416] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:44:09.601810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc3e9e-3ef2-4768-9ad7-dd915aa6bb8d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:09.603252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-98fc3e9e-3ef2-4768-9ad7-dd915aa6bb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.746772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fc3e9e-3ef2-4768-9ad7-dd915aa6bb8d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:09.788085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:09.862185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:09.863154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:09.863643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:09.864132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:09.866492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:09.963411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:09.965013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254800d-dd0a-471a-a5fd-353a04a4a4d7 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:09.982546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 715/1417] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:11.106775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a11d4b-db88-4499-a585-23fa00fabefd None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.107298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a11d4b-db88-4499-a585-23fa00fabefd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.107298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a11d4b-db88-4499-a585-23fa00fabefd None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.108601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 703/1418] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:44:11.114591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7e1e4-7db0-4270-871a-4cb105509741 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.115013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7e1e4-7db0-4270-871a-4cb105509741 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.115126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7e1e4-7db0-4270-871a-4cb105509741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.168869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:11.191170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-84d7e1e4-7db0-4270-871a-4cb105509741 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:11.192424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-84d7e1e4-7db0-4270-871a-4cb105509741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:11.288633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 716/1419] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:11.439140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a11d4b-db88-4499-a585-23fa00fabefd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:11.440603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d3a11d4b-db88-4499-a585-23fa00fabefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:11.591364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a11d4b-db88-4499-a585-23fa00fabefd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:11.603183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:11.703665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:11.704187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:11.704464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:11.704734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:11.706368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:11.796808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:11.798434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7568f574-5769-4d00-ba0b-c7c440463ab0 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:11.816587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 704/1420] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:12.916945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a16c-f954-4c38-86ed-32bae672efc6 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.917478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a16c-f954-4c38-86ed-32bae672efc6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.917478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920a16c-f954-4c38-86ed-32bae672efc6 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:12.918871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 717/1421] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:12.925137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207f7d-b756-4fbc-8728-f5608dcd94e3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:12.925592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207f7d-b756-4fbc-8728-f5608dcd94e3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:12.925752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06207f7d-b756-4fbc-8728-f5608dcd94e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:12.945718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:13.003133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-06207f7d-b756-4fbc-8728-f5608dcd94e3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:13.004336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-06207f7d-b756-4fbc-8728-f5608dcd94e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:13.099544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 705/1422] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:13.253082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a16c-f954-4c38-86ed-32bae672efc6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:13.254645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8920a16c-f954-4c38-86ed-32bae672efc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:13.369368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:13.404763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920a16c-f954-4c38-86ed-32bae672efc6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:13.517094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:13.517695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:13.518001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:13.518271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:13.519878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:13.596144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:13.611033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:13.612655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a108b043-d260-4a34-8180-21fbbe813202 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:13.630814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 718/1423] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:14.737564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a34e72-002a-49a3-a5cc-80a416e6228a None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.738182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a34e72-002a-49a3-a5cc-80a416e6228a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.738182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a34e72-002a-49a3-a5cc-80a416e6228a None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.739501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 706/1424] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:14.745780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c84a2-35b4-4051-8549-31a15ef2c636 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:14.746200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c84a2-35b4-4051-8549-31a15ef2c636 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:14.746328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30c84a2-35b4-4051-8549-31a15ef2c636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:14.822106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b30c84a2-35b4-4051-8549-31a15ef2c636 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:14.823325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b30c84a2-35b4-4051-8549-31a15ef2c636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:14.917751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 719/1425] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:15.068992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a34e72-002a-49a3-a5cc-80a416e6228a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:15.070543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-40a34e72-002a-49a3-a5cc-80a416e6228a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:15.152397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.213089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a34e72-002a-49a3-a5cc-80a416e6228a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:15.326008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:15.326550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:15.326850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:15.327115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:15.328739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:15.377061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:15.418655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:15.420228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d51b2f6-1279-4fee-8f36-cb210170b50b service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:15.437746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 707/1426] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:16.487745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d482d12-840b-4ccc-ae32-366c40e5772b None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.487745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d482d12-840b-4ccc-ae32-366c40e5772b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.488209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d482d12-840b-4ccc-ae32-366c40e5772b None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.489350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 720/1427] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:16.496720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb046-091c-4db5-98e9-63714179ea86 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:16.497332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb046-091c-4db5-98e9-63714179ea86 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:16.497332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3fb046-091c-4db5-98e9-63714179ea86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:16.574372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-be3fb046-091c-4db5-98e9-63714179ea86 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:16.575643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-be3fb046-091c-4db5-98e9-63714179ea86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:16.672008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 708/1428] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:16.822942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d482d12-840b-4ccc-ae32-366c40e5772b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:16.824595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8d482d12-840b-4ccc-ae32-366c40e5772b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:16.873498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:16.969988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d482d12-840b-4ccc-ae32-366c40e5772b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:17.088510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:17.089024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:17.089300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:17.089567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:17.091823 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:17.098581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:17.178632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:17.180207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a940890-ace7-4f6f-9b50-311715997b3a service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:17.198235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 721/1429] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:18.259040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a04da9-c409-42f8-b1d4-2e55cd97c478 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.259592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a04da9-c409-42f8-b1d4-2e55cd97c478 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.259592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a04da9-c409-42f8-b1d4-2e55cd97c478 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.260915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 709/1430] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:18.267139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf78925-4da7-40e2-b5fd-d4ed858a1e1d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.267597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf78925-4da7-40e2-b5fd-d4ed858a1e1d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.267731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf78925-4da7-40e2-b5fd-d4ed858a1e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.343369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8cf78925-4da7-40e2-b5fd-d4ed858a1e1d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:18.344578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8cf78925-4da7-40e2-b5fd-d4ed858a1e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:18.410199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:18.439893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 722/1431] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:18.589721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a04da9-c409-42f8-b1d4-2e55cd97c478 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:18.591913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f9a04da9-c409-42f8-b1d4-2e55cd97c478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:18.609025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:18.735242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a04da9-c409-42f8-b1d4-2e55cd97c478 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:18.820019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:18.846175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:18.846770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:18.847041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:18.847305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:18.848869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:18.937140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:18.938741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafcd5b8-97f4-45f2-8908-64ff9f2c8022 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:18.957121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 710/1432] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:20.007111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75921d0-1a09-49d9-8699-1826c4b6141c None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.007697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75921d0-1a09-49d9-8699-1826c4b6141c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.007697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75921d0-1a09-49d9-8699-1826c4b6141c None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.009044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 723/1433] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:20.016064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c963fe-0b46-430c-b1aa-d57b48ea0977 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.016561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c963fe-0b46-430c-b1aa-d57b48ea0977 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.016682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c963fe-0b46-430c-b1aa-d57b48ea0977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.092882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f4c963fe-0b46-430c-b1aa-d57b48ea0977 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:20.094043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f4c963fe-0b46-430c-b1aa-d57b48ea0977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:20.134228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:20.189172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 711/1434] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:20.338210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75921d0-1a09-49d9-8699-1826c4b6141c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:20.339770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e75921d0-1a09-49d9-8699-1826c4b6141c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:20.483979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75921d0-1a09-49d9-8699-1826c4b6141c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:20.539150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:20.599658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:20.600260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:20.600538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:20.600808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:20.602453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:20.690164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:20.691756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be625f2-1aeb-4a94-a219-1113b799ca06 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:20.709754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 724/1435] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:21.760186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3395be7-2a9d-4777-af29-81c94c07e02e None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.760760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3395be7-2a9d-4777-af29-81c94c07e02e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.760760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3395be7-2a9d-4777-af29-81c94c07e02e None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.762096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 712/1436] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:44:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:21.768803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3877-ed9a-4c31-bf79-0113df3ea4cc None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:21.769261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3877-ed9a-4c31-bf79-0113df3ea4cc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:21.769372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee3877-ed9a-4c31-bf79-0113df3ea4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:21.844398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bbee3877-ed9a-4c31-bf79-0113df3ea4cc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:21.845628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bbee3877-ed9a-4c31-bf79-0113df3ea4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:21.847703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:21.941316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 725/1437] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:44:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:22.090678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3395be7-2a9d-4777-af29-81c94c07e02e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:22.092139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f3395be7-2a9d-4777-af29-81c94c07e02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:22.234648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3395be7-2a9d-4777-af29-81c94c07e02e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:22.261129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:22.344788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:22.345318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:22.345590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:22.345888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:22.347469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:22.435536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:22.437124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e57430f-2750-43ff-8bdf-b42e065ee488 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:22.454644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 713/1438] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:24.493589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72212c0e-a9a1-4d6f-b368-dd94cb37d432 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.494172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72212c0e-a9a1-4d6f-b368-dd94cb37d432 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.494172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72212c0e-a9a1-4d6f-b368-dd94cb37d432 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.495795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 726/1439] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:24.502648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58deb97-e598-4e45-8965-404de40c67b1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:24.503093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58deb97-e598-4e45-8965-404de40c67b1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:24.503213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58deb97-e598-4e45-8965-404de40c67b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:24.543501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:24.570711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d58deb97-e598-4e45-8965-404de40c67b1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:24.571911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d58deb97-e598-4e45-8965-404de40c67b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:24.667974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 714/1440] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:24.952494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72212c0e-a9a1-4d6f-b368-dd94cb37d432 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:24.954096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-72212c0e-a9a1-4d6f-b368-dd94cb37d432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:25.068097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72212c0e-a9a1-4d6f-b368-dd94cb37d432 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:25.100589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:25.141042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:25.141641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:25.141964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:25.142219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:25.143807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:25.250031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:25.251475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43169557-12c3-401d-829d-b24cf7d68b4e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:25.269477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 727/1441] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:27.547642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d582-e113-41f4-84ba-2502dc930a8d None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.548223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d582-e113-41f4-84ba-2502dc930a8d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.548223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406d582-e113-41f4-84ba-2502dc930a8d None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.549494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 715/1442] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:27.555008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75310766-7e17-4c4d-8f55-855264de7b9a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:27.555442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75310766-7e17-4c4d-8f55-855264de7b9a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:27.555552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75310766-7e17-4c4d-8f55-855264de7b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:27.620372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-75310766-7e17-4c4d-8f55-855264de7b9a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:27.621595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-75310766-7e17-4c4d-8f55-855264de7b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:27.634870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:27.716433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 728/1443] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:27.998578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d582-e113-41f4-84ba-2502dc930a8d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:28.000100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f406d582-e113-41f4-84ba-2502dc930a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:28.101844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406d582-e113-41f4-84ba-2502dc930a8d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:28.114412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:28.175381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:28.175979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:28.176253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:28.176522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:28.178140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:28.284908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:28.286342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42090ebf-59e2-4dbe-b28d-0e908e9c5af6 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:28.304017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 716/1444] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:30.772851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aab76-a36e-49e4-9565-9af0af6864db None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.773437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aab76-a36e-49e4-9565-9af0af6864db None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.773437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6aab76-a36e-49e4-9565-9af0af6864db None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.774764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 729/1445] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:30.780070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50f84-6974-4e1c-9653-447d5551cc61 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:30.780518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50f84-6974-4e1c-9653-447d5551cc61 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:30.780637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b50f84-6974-4e1c-9653-447d5551cc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:30.829542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:30.848425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-07b50f84-6974-4e1c-9653-447d5551cc61 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:30.849653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-07b50f84-6974-4e1c-9653-447d5551cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:30.945171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 717/1446] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:31.224507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6aab76-a36e-49e4-9565-9af0af6864db None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:31.226140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9b6aab76-a36e-49e4-9565-9af0af6864db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:31.329274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6aab76-a36e-49e4-9565-9af0af6864db None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:31.371464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:31.404748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:31.405346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:31.405620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:31.405918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:31.407526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:31.514793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:31.516218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb47deb-5bf9-43f0-acbb-950ef2997e7e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:31.534083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 730/1447] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:44:33.836579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6dd34-b20f-49bf-8e6c-2448a0adf370 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.837134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6dd34-b20f-49bf-8e6c-2448a0adf370 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.837134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e6dd34-b20f-49bf-8e6c-2448a0adf370 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.838506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 718/1448] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:33.844160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dd5cc-2048-42f8-9260-0cce85fb0e46 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:33.844561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dd5cc-2048-42f8-9260-0cce85fb0e46 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:33.844709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72dd5cc-2048-42f8-9260-0cce85fb0e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:33.910489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c72dd5cc-2048-42f8-9260-0cce85fb0e46 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:33.911707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c72dd5cc-2048-42f8-9260-0cce85fb0e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:33.932197 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.008372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 731/1449] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:34.314216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f169af-c644-4d1c-8afc-06a89655909c None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.314480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f169af-c644-4d1c-8afc-06a89655909c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.314753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f169af-c644-4d1c-8afc-06a89655909c None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.316114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 719/1450] 10.222.0.15 () {54 vars in 965 bytes} [Tue Sep 13 19:44:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:34.321937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123e0f-8677-4b8d-a9e7-8f966211069a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.322225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123e0f-8677-4b8d-a9e7-8f966211069a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.322501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f123e0f-8677-4b8d-a9e7-8f966211069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.397572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7f123e0f-8677-4b8d-a9e7-8f966211069a None None] MFA Rules not processed for user `36fab0873915497cada62e8ea73bd4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:34.398806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7f123e0f-8677-4b8d-a9e7-8f966211069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:34.473968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 732/1451] 10.222.0.15 () {58 vars in 1062 bytes} [Tue Sep 13 19:44:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:34.483555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f169af-c644-4d1c-8afc-06a89655909c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:34.485076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-90f169af-c644-4d1c-8afc-06a89655909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:34.589160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f169af-c644-4d1c-8afc-06a89655909c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:34.610552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:34.663670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:34.664263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:34.664533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:34.665050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:34.667086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:34.772947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:34.774390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a66a5f-a732-42fb-8316-116ba05b2230 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:34.792403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 720/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:37.458790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f86167-d9b4-4309-8c19-2276c3cb589c None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:37.459332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f86167-d9b4-4309-8c19-2276c3cb589c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:37.459470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f86167-d9b4-4309-8c19-2276c3cb589c None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:37.460681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 733/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:37.466725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39639d-7391-4bb4-8300-5104cc3112af None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:37.467138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39639d-7391-4bb4-8300-5104cc3112af None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:37.467251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd39639d-7391-4bb4-8300-5104cc3112af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:37.529590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:37.532318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-fd39639d-7391-4bb4-8300-5104cc3112af None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:37.534059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-fd39639d-7391-4bb4-8300-5104cc3112af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:37.629677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 721/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:37.913657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f86167-d9b4-4309-8c19-2276c3cb589c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:37.915466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-55f86167-d9b4-4309-8c19-2276c3cb589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:37.933905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:38.017267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f86167-d9b4-4309-8c19-2276c3cb589c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:38.087649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:38.088287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:38.088512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:38.088778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:38.090932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:38.160130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:38.198992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:38.201004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d55b0a-de88-42dc-a9c9-a74ce9f9f739 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:38.218290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 734/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:40.541521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d467-13d5-4fb4-b598-ff1e99e310c0 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.542134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d467-13d5-4fb4-b598-ff1e99e310c0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.542134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb61d467-13d5-4fb4-b598-ff1e99e310c0 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.543421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 722/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:40.549369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb81707-d7a9-4ebb-a10f-555360f72f45 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:40.549789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb81707-d7a9-4ebb-a10f-555360f72f45 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:40.549971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb81707-d7a9-4ebb-a10f-555360f72f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:40.615042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-afb81707-d7a9-4ebb-a10f-555360f72f45 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:40.616241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-afb81707-d7a9-4ebb-a10f-555360f72f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:40.713269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 735/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:41.002107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d467-13d5-4fb4-b598-ff1e99e310c0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:41.003633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bb61d467-13d5-4fb4-b598-ff1e99e310c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:41.219924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb61d467-13d5-4fb4-b598-ff1e99e310c0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:41.248309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:41.294171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:41.294774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:41.295046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:41.295315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:41.296904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:41.405229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:41.406655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40ab8e-139a-436f-95b8-b709e800923b service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:41.424410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 723/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:43.637967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110aa0-87ad-440d-8651-b8092002a884 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.638517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110aa0-87ad-440d-8651-b8092002a884 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.638517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f110aa0-87ad-440d-8651-b8092002a884 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.639822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 736/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:43.646240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7b01f-d7de-4325-b639-9b4a43c2eee8 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:43.646698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7b01f-d7de-4325-b639-9b4a43c2eee8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:43.646795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7b01f-d7de-4325-b639-9b4a43c2eee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:43.712558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f9d7b01f-d7de-4325-b639-9b4a43c2eee8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:43.714205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f9d7b01f-d7de-4325-b639-9b4a43c2eee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:43.723673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:43.810010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 724/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:44.102261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110aa0-87ad-440d-8651-b8092002a884 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:44.103804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-6f110aa0-87ad-440d-8651-b8092002a884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:44.132376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.209297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f110aa0-87ad-440d-8651-b8092002a884 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:44.283748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:44.284341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:44.284613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:44.284878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:44.286480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:44.359844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:44.392819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:44.394272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c354d0-1e7e-44ea-a791-a8b1941ff400 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:44.411736 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 737/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:46.608528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c22e77-3f0a-447b-9084-2eaead540110 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.609128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c22e77-3f0a-447b-9084-2eaead540110 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.609128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c22e77-3f0a-447b-9084-2eaead540110 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.610424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 725/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:46.615594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc975d66-70c1-4b8f-a608-c73411f093ff None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:46.616037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc975d66-70c1-4b8f-a608-c73411f093ff None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:46.616140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc975d66-70c1-4b8f-a608-c73411f093ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:46.681039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cc975d66-70c1-4b8f-a608-c73411f093ff None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:46.682267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cc975d66-70c1-4b8f-a608-c73411f093ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:46.779575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 738/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:47.058494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c22e77-3f0a-447b-9084-2eaead540110 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:47.060659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-66c22e77-3f0a-447b-9084-2eaead540110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:47.162517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c22e77-3f0a-447b-9084-2eaead540110 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:47.199913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:47.236237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:47.236836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:47.237114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:47.237386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:47.239021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:47.345063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:47.346433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99398713-b884-45a3-8ce6-9cd6af0874e6 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:47.364421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 726/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:49.509056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b78a56-a90a-4e43-88e6-df9fa62db446 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.509621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b78a56-a90a-4e43-88e6-df9fa62db446 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.509621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b78a56-a90a-4e43-88e6-df9fa62db446 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.510972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 739/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:49.516198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc02971-0c99-4be6-90c8-47e310becaed None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:49.516658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc02971-0c99-4be6-90c8-47e310becaed None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:49.516765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc02971-0c99-4be6-90c8-47e310becaed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:49.580883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4cc02971-0c99-4be6-90c8-47e310becaed None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:49.582102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4cc02971-0c99-4be6-90c8-47e310becaed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:49.600061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:49.676956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 727/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:49.958598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b78a56-a90a-4e43-88e6-df9fa62db446 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:49.960145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-49b78a56-a90a-4e43-88e6-df9fa62db446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:49.997512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.063688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b78a56-a90a-4e43-88e6-df9fa62db446 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:50.135266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:50.135830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:50.136105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:50.136371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:50.138000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:50.219157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:50.245786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:50.247211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01a057-5f20-45b7-be96-af9075d341ed service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:50.264628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 740/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:44:52.417887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8cd4c-5252-4977-9ca4-ba031a49fa41 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.418446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8cd4c-5252-4977-9ca4-ba031a49fa41 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.418446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8cd4c-5252-4977-9ca4-ba031a49fa41 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.419781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 728/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:44:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:44:52.425186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34729c-ab4c-4198-b534-2e4516b46d75 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:52.425593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34729c-ab4c-4198-b534-2e4516b46d75 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:52.425766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df34729c-ab4c-4198-b534-2e4516b46d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:52.490690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-df34729c-ab4c-4198-b534-2e4516b46d75 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:44:52.491925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-df34729c-ab4c-4198-b534-2e4516b46d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:52.589665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 741/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:44:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:44:52.875811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8cd4c-5252-4977-9ca4-ba031a49fa41 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:44:52.877400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-66e8cd4c-5252-4977-9ca4-ba031a49fa41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:52.979098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8cd4c-5252-4977-9ca4-ba031a49fa41 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:44:53.026134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:44:53.052332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:44:53.052923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:44:53.053203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:44:53.053474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:44:53.055078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:44:53.161175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:44:53.162523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8cd90-232b-43dd-8e79-e9e32272f3ea service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:44:53.180456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 729/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:44:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:02.588132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=81 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:03.178319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=66 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:12.939283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c047-70d7-4132-9c3b-b78d3a93f851 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.939854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c047-70d7-4132-9c3b-b78d3a93f851 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.939854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b299c047-70d7-4132-9c3b-b78d3a93f851 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:12.941160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 742/1471] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:12.946648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b440-ee61-43e5-866b-da1e00d6f7e1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:12.947108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b440-ee61-43e5-866b-da1e00d6f7e1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:12.947212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32b440-ee61-43e5-866b-da1e00d6f7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.012811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7f32b440-ee61-43e5-866b-da1e00d6f7e1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:13.014040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7f32b440-ee61-43e5-866b-da1e00d6f7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:13.108672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 730/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:13.419600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62543d-7793-4bb4-a6fa-40491d9f79bf None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.419875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62543d-7793-4bb4-a6fa-40491d9f79bf None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.420151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc62543d-7793-4bb4-a6fa-40491d9f79bf None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.421463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 743/1473] 10.222.0.15 () {54 vars in 966 bytes} [Tue Sep 13 19:45:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:13.426493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500401b-76e0-4bb3-a0e8-60c6d10ca34d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.426775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500401b-76e0-4bb3-a0e8-60c6d10ca34d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.427042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500401b-76e0-4bb3-a0e8-60c6d10ca34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.504503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4500401b-76e0-4bb3-a0e8-60c6d10ca34d None None] MFA Rules not processed for user `c21dea41dba24bf88466a7c3f00b8c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:13.505693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4500401b-76e0-4bb3-a0e8-60c6d10ca34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:13.525509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.582506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 731/1474] 10.222.0.15 () {58 vars in 1063 bytes} [Tue Sep 13 19:45:13 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:13.590834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62543d-7793-4bb4-a6fa-40491d9f79bf None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:13.592372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cc62543d-7793-4bb4-a6fa-40491d9f79bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:13.693863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc62543d-7793-4bb4-a6fa-40491d9f79bf None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:13.884627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:13.885209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:13.885486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:13.885767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:13.887860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:13.940505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:13.995074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:13.996507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c896ab-ca5f-41cb-a855-fed214eba27f service glance] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:14.014416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 744/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:16.259946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1ab10-a5d7-4dfb-9410-78b927add4a6 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.260498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1ab10-a5d7-4dfb-9410-78b927add4a6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.260498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b1ab10-a5d7-4dfb-9410-78b927add4a6 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.261875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 732/1476] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:16.267109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c647a6e-b91d-4732-a97b-b05d28702d3f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.267567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c647a6e-b91d-4732-a97b-b05d28702d3f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.267678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c647a6e-b91d-4732-a97b-b05d28702d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.331119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9c647a6e-b91d-4732-a97b-b05d28702d3f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:16.332303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9c647a6e-b91d-4732-a97b-b05d28702d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:16.424868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 745/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:16.701176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1ab10-a5d7-4dfb-9410-78b927add4a6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:16.702910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-38b1ab10-a5d7-4dfb-9410-78b927add4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:16.829080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b1ab10-a5d7-4dfb-9410-78b927add4a6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:16.878292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:16.923297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:16.923888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:16.924161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:16.924425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:16.925052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:16.926796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:45:16.926796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:45:16.926796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:45:16.927915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37ca08-4cbb-4ba1-9efd-d89abd74acd2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:16.946266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 733/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Tue Sep 13 19:45:16 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:45:19.098215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8199-2888-4143-89d1-cf4820da1d37 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.098757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8199-2888-4143-89d1-cf4820da1d37 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.098757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbf8199-2888-4143-89d1-cf4820da1d37 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.099991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 746/1479] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:19.105803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158d78-a0f0-4269-b3f4-79e22b09acc5 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.106242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158d78-a0f0-4269-b3f4-79e22b09acc5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.106346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158d78-a0f0-4269-b3f4-79e22b09acc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.171855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1c158d78-a0f0-4269-b3f4-79e22b09acc5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:19.173055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1c158d78-a0f0-4269-b3f4-79e22b09acc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:19.272122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 734/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:19.563998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf8199-2888-4143-89d1-cf4820da1d37 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:19.565510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2cbf8199-2888-4143-89d1-cf4820da1d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:19.576561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.710985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbf8199-2888-4143-89d1-cf4820da1d37 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:19.788037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:19.819859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:19.820384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:19.820656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:19.820914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:19.822554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:19.909127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:19.910722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8a875b-8298-409e-8a0b-629554750d32 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:19.928085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 747/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:22.240845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571153e6-4ecf-49ee-8829-6665c6aca3d5 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.241596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571153e6-4ecf-49ee-8829-6665c6aca3d5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.241803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571153e6-4ecf-49ee-8829-6665c6aca3d5 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.243357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 735/1482] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:22.249095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b57e4a-cd3b-47a3-b7c4-161959d4a523 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.249552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b57e4a-cd3b-47a3-b7c4-161959d4a523 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.249712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b57e4a-cd3b-47a3-b7c4-161959d4a523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.318867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-44b57e4a-cd3b-47a3-b7c4-161959d4a523 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:22.320082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-44b57e4a-cd3b-47a3-b7c4-161959d4a523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:22.364275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.413629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 748/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:22.701796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571153e6-4ecf-49ee-8829-6665c6aca3d5 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:22.703351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-571153e6-4ecf-49ee-8829-6665c6aca3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:22.728535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.806702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571153e6-4ecf-49ee-8829-6665c6aca3d5 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:22.880122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:22.880719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:22.880996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:22.881267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:22.882891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:22.956347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:22.991172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:22.992584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ec4787-e17a-4b87-87bc-d7c0ef3892c0 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:23.010127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 736/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:25.202636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50286cf-5492-4718-827a-09bf52e3e7d6 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.203176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50286cf-5492-4718-827a-09bf52e3e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.203176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50286cf-5492-4718-827a-09bf52e3e7d6 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.204463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 749/1485] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:25.209574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658778d-58e8-40b5-b9c7-09ca41e1f24c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.210044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658778d-58e8-40b5-b9c7-09ca41e1f24c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.210184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7658778d-58e8-40b5-b9c7-09ca41e1f24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.274636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7658778d-58e8-40b5-b9c7-09ca41e1f24c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:25.275820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7658778d-58e8-40b5-b9c7-09ca41e1f24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:25.375866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 737/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:25.670428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50286cf-5492-4718-827a-09bf52e3e7d6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:25.671971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f50286cf-5492-4718-827a-09bf52e3e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:25.772922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50286cf-5492-4718-827a-09bf52e3e7d6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:25.846285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:25.847888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:25.848536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:25.848995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:25.849466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:25.851505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:25.957796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:25.959122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d430027d-0fae-4260-acae-b7faffb3c6ab service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:25.981324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 750/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:28.212007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012157c-f249-4497-b5f2-0790bb4adf05 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:28.212436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012157c-f249-4497-b5f2-0790bb4adf05 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:28.212561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012157c-f249-4497-b5f2-0790bb4adf05 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:28.213936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 738/1488] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:28.219183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be006a21-54cb-4392-96f5-f27d3e734b95 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:28.219607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be006a21-54cb-4392-96f5-f27d3e734b95 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:28.219755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be006a21-54cb-4392-96f5-f27d3e734b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:28.297886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-be006a21-54cb-4392-96f5-f27d3e734b95 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:28.299142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-be006a21-54cb-4392-96f5-f27d3e734b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.360571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:28.404092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 751/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:28.703895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012157c-f249-4497-b5f2-0790bb4adf05 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:28.705411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c012157c-f249-4497-b5f2-0790bb4adf05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.742531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:28.811625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012157c-f249-4497-b5f2-0790bb4adf05 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:28.883936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:28.884528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:28.884804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:28.885073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:28.887214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:28.965091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:28.993207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:28.994637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988a77bb-a520-49f7-a426-ae3480dba019 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:29.012569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 739/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:31.336673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eea50-0ceb-40b7-bf1a-8bbab8efc4d1 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.337242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eea50-0ceb-40b7-bf1a-8bbab8efc4d1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.337242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454eea50-0ceb-40b7-bf1a-8bbab8efc4d1 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.338671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 752/1491] 10.222.0.15 () {54 vars in 927 bytes} [Tue Sep 13 19:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:45:31.343962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12c350-2a6e-423d-a84e-9f97bd60b3d2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.344390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12c350-2a6e-423d-a84e-9f97bd60b3d2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.344493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed12c350-2a6e-423d-a84e-9f97bd60b3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.409227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ed12c350-2a6e-423d-a84e-9f97bd60b3d2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:45:31.410446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ed12c350-2a6e-423d-a84e-9f97bd60b3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:31.506023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 740/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Tue Sep 13 19:45:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:45:31.799585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eea50-0ceb-40b7-bf1a-8bbab8efc4d1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:45:31.801146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-454eea50-0ceb-40b7-bf1a-8bbab8efc4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:31.902965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454eea50-0ceb-40b7-bf1a-8bbab8efc4d1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:45:31.974529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:45:31.975119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:45:31.975394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:45:31.975659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:45:31.977235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:45:31.985584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:32.083370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:45:32.084749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240e4dd9-96a3-4800-97e6-3b3860e783c7 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:45:32.102974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 753/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:45:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:45:41.504122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=82 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:45:42.100714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=50 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:44.295730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:44.296260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:44.296260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:44.371749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:44.372952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:44.466725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 741/1494] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:44.475001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:44.475521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:44.475521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:44.548885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:44.550040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:44.643383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 754/1495] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:44.647944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:44.649127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:44.660273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e633d-9a35-4a9f-bb02-acaa1eccd40b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:44.720535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:44.754239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:44.754837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:44.755109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:44.755381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:44.756032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:46:44.756343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:44.757320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:44.757320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:46:44.757320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:46:44.758228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:44.769254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 742/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:46:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:46:44.777508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:44.786480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1981e16-ffe3-40e6-ac4d-02bdee3da4c0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:44.847375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:44.881338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:44.881965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:44.882243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:44.882514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:44.883137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:44.884872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:44.884872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:46:44.884872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:46:44.886090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:44.941922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 755/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:46:44 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:44.946083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:44.947596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:44.981289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa614615-c887-45fb-a801-c77e54ba7f22 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.077584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.078211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.078486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.078753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.079395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.080392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:45.080392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:46:45.080392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:46:45.081591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.123680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 743/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:46:44 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:46:45.127949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.129470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:45.190156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570fff1-1231-4276-a193-ee8151f331a8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.270352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.281238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.281865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.282136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.282401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.283031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.284032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:45.284032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:46:45.284032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:46:45.284906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.297706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 756/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:46:45.301710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.303270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:45.315117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32b915d-108d-40bf-89c5-55ace6ab7d6e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.341270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.410209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.410706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.410977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.411243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.411884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.412924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:45.412924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:46:45.412924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:46:45.414287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01f77b-cfe5-4439-ab74-6685b0d73748 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.427750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 744/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:46:45.431997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.433555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:45.467349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f35e53-8af0-40df-8b51-2e6bb3356414 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:45.559040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:45.559705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.559949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.560216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] PATH_INFO: `/v3/projects/c6bb820156844d209ae5d0bda96936e0/users/77d23de77f9844cdb28c95c2a6d3cd7e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.582349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bb820156844d209ae5d0bda96936e0, user_id=77d23de77f9844cdb28c95c2a6d3cd7e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:45.583361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:45.583361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:46:45.583361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:46:45.585900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4158d84-00a4-45a2-9657-767061355619 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:45.609709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 757/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:46:45 2022] PUT /identity/v3/projects/c6bb820156844d209ae5d0bda96936e0/users/77d23de77f9844cdb28c95c2a6d3cd7e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:46:45.622033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389401-e698-423a-b2b5-e8d4f84b0997 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.622313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389401-e698-423a-b2b5-e8d4f84b0997 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.622578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1389401-e698-423a-b2b5-e8d4f84b0997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.680775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c1389401-e698-423a-b2b5-e8d4f84b0997 None None] MFA Rules not processed for user `77d23de77f9844cdb28c95c2a6d3cd7e`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:45.681935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c1389401-e698-423a-b2b5-e8d4f84b0997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:45.757713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 745/1502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:46:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:46:45.770963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7927f6-0429-458e-9922-dc8dfbd1a11b None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.771248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7927f6-0429-458e-9922-dc8dfbd1a11b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.771518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7927f6-0429-458e-9922-dc8dfbd1a11b None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.772959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 758/1503] 10.222.0.15 () {54 vars in 964 bytes} [Tue Sep 13 19:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:45.777945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294dc4b-4dbf-4173-8c83-0a00f6a45e12 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:45.778228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294dc4b-4dbf-4173-8c83-0a00f6a45e12 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:45.778496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c294dc4b-4dbf-4173-8c83-0a00f6a45e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:45.810246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:45.854882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c294dc4b-4dbf-4173-8c83-0a00f6a45e12 None None] MFA Rules not processed for user `0f8ea3cf81f944a2b02e59b3ff049f6d`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:45.856090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c294dc4b-4dbf-4173-8c83-0a00f6a45e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:45.931955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 746/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Tue Sep 13 19:46:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:45.939825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7927f6-0429-458e-9922-dc8dfbd1a11b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:45.941423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2f7927f6-0429-458e-9922-dc8dfbd1a11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:45.989989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:46.042858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7927f6-0429-458e-9922-dc8dfbd1a11b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:46.115606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:46.116740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:46.117013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:46.117280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:46.118890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:46.213234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:46.214604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:46.217985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:46.232658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 759/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:46:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:46:46.278163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b2848-ca01-45e9-a2c5-f7108ba13659 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:46.278435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b2848-ca01-45e9-a2c5-f7108ba13659 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:46.278708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b2848-ca01-45e9-a2c5-f7108ba13659 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:46.279958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 747/1506] 10.222.0.15 () {52 vars in 830 bytes} [Tue Sep 13 19:46:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:46:46.283530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:46.284778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:46.382065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a258bc-b871-4254-a8e9-f5ff71053717 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:46.478529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:46.479080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:46.479357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:46.479625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] PATH_INFO: `/v3/users/77d23de77f9844cdb28c95c2a6d3cd7e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:46.492204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] RBAC: Authorizing `identity:delete_user(user_id=77d23de77f9844cdb28c95c2a6d3cd7e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:46.493229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:46.493229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:46:46.493229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:46:46.494461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:46.503893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:46.562321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77d23de77f9844cdb28c95c2a6d3cd7e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:46.571164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77d23de77f9844cdb28c95c2a6d3cd7e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:46.571658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77d23de77f9844cdb28c95c2a6d3cd7e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:46.572081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77d23de77f9844cdb28c95c2a6d3cd7e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:46.586290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77d23de77f9844cdb28c95c2a6d3cd7e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:46.613716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 760/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:46:46 2022] DELETE /identity/v3/users/77d23de77f9844cdb28c95c2a6d3cd7e => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:46.618109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:46.618582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:46.618691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:46.675726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:46:46.676926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:46.756176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:46.768467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 748/1508] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:46:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:46:46.778568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:46.779808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:46.881086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce83c37c-ea60-466a-a939-f7ebb853290c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:46.940470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:46.962724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:46.963321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:46.963601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:46.963868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:46.965448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:47.071769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:47.073192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bed828d-f042-4057-94e2-78333b3efebf service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:47.091663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 761/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:46:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:46:47.438881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:46:47.440160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:46:47.451989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4d6c12-09f6-4807-a3fb-741731d32235 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:46:47.551160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:46:47.551781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:46:47.552053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:46:47.552317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] PATH_INFO: `/v3/projects/c6bb820156844d209ae5d0bda96936e0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:46:47.562525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6bb820156844d209ae5d0bda96936e0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:46:47.563541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:46:47.563541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:46:47.563541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:46:47.564719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:46:47.623506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6bb820156844d209ae5d0bda96936e0', 'request_id': 'req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:47.636949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6bb820156844d209ae5d0bda96936e0', 'request_id': 'req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:47.647034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6bb820156844d209ae5d0bda96936e0', 'request_id': 'req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:47.669592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] The token cache is being invalidate because project c6bb820156844d209ae5d0bda96936e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:46:47.669904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:47.670324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b7173ef1-7fdd-45d0-a0d6-3442903e6f5b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:46:47.671995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 749/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:46:47 2022] DELETE /identity/v3/projects/c6bb820156844d209ae5d0bda96936e0 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:46:57.089431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=62 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:46:57.667632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:18.323841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933322-35ff-448c-b460-4d8a6e6486bb None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:18.324337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933322-35ff-448c-b460-4d8a6e6486bb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:18.324446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4933322-35ff-448c-b460-4d8a6e6486bb None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:18.325753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 762/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:18.330571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19273cf-30ad-4c24-9937-9550ab49190f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:18.331022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19273cf-30ad-4c24-9937-9550ab49190f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:18.331128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19273cf-30ad-4c24-9937-9550ab49190f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:18.409848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d19273cf-30ad-4c24-9937-9550ab49190f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:18.411038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d19273cf-30ad-4c24-9937-9550ab49190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:18.518263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 750/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:18.675602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933322-35ff-448c-b460-4d8a6e6486bb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:18.677703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d4933322-35ff-448c-b460-4d8a6e6486bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:18.800144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4933322-35ff-448c-b460-4d8a6e6486bb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:18.805489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c74635-cab6-4c4d-8a29-971353552029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:18.806121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74635-cab6-4c4d-8a29-971353552029 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:18.806400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74635-cab6-4c4d-8a29-971353552029 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:18.806669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c74635-cab6-4c4d-8a29-971353552029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:18.814705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74635-cab6-4c4d-8a29-971353552029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:18.816296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c74635-cab6-4c4d-8a29-971353552029 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:18.825903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 763/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:19.901035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925afc-49a6-4905-a656-68a84a25f422 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.901583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925afc-49a6-4905-a656-68a84a25f422 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.901583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58925afc-49a6-4905-a656-68a84a25f422 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.902967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 751/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:19.909505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5503f-60ed-4d08-8844-7e8d4f1a8923 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:19.909968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5503f-60ed-4d08-8844-7e8d4f1a8923 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:19.910108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5503f-60ed-4d08-8844-7e8d4f1a8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:19.929985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8ec5503f-60ed-4d08-8844-7e8d4f1a8923 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:19.931175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8ec5503f-60ed-4d08-8844-7e8d4f1a8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:19.938766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 764/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:20.087897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925afc-49a6-4905-a656-68a84a25f422 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:20.102777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58925afc-49a6-4905-a656-68a84a25f422 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:20.108392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d66ba66-b352-43e9-b44d-25f5ef4440b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:20.108990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ba66-b352-43e9-b44d-25f5ef4440b5 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.109262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ba66-b352-43e9-b44d-25f5ef4440b5 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.109527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d66ba66-b352-43e9-b44d-25f5ef4440b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.118694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66ba66-b352-43e9-b44d-25f5ef4440b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:20.120245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d66ba66-b352-43e9-b44d-25f5ef4440b5 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:20.129328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 752/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:20.134216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f44ff7-31f1-47ed-874d-2dfc033a957e None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.134870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f44ff7-31f1-47ed-874d-2dfc033a957e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.134870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f44ff7-31f1-47ed-874d-2dfc033a957e None None] PATH_INFO: `/v3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.135975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 765/1517] 10.222.0.61 () {54 vars in 925 bytes} [Tue Sep 13 19:48:20 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:48:20.141980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5bbc-ad93-4d91-a985-dd1d9e5dbfd5 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:20.142267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5bbc-ad93-4d91-a985-dd1d9e5dbfd5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:20.142543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625c5bbc-ad93-4d91-a985-dd1d9e5dbfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:20.207867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-625c5bbc-ad93-4d91-a985-dd1d9e5dbfd5 None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:20.209100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-625c5bbc-ad93-4d91-a985-dd1d9e5dbfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:20.227382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:20.288137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 753/1518] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Sep 13 19:48:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:48:29.147789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc065d7-fe97-4e1c-8fa6-e770a8bf50c4 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.148266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc065d7-fe97-4e1c-8fa6-e770a8bf50c4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.148372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc065d7-fe97-4e1c-8fa6-e770a8bf50c4 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.149617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 766/1519] 10.222.0.15 () {54 vars in 961 bytes} [Tue Sep 13 19:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:29.154597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311621e8-c60d-499b-a495-2f358dfcf99c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.155018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311621e8-c60d-499b-a495-2f358dfcf99c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.155196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311621e8-c60d-499b-a495-2f358dfcf99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.175255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-311621e8-c60d-499b-a495-2f358dfcf99c None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:29.176442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-311621e8-c60d-499b-a495-2f358dfcf99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:29.184825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 754/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Sep 13 19:48:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:29.192801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc065d7-fe97-4e1c-8fa6-e770a8bf50c4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:29.205688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc065d7-fe97-4e1c-8fa6-e770a8bf50c4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:29.210867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec4d53-e443-4305-ab7d-4ccd67a91c48 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:29.211464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec4d53-e443-4305-ab7d-4ccd67a91c48 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:29.211742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec4d53-e443-4305-ab7d-4ccd67a91c48 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:29.212016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec4d53-e443-4305-ab7d-4ccd67a91c48 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:29.220405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec4d53-e443-4305-ab7d-4ccd67a91c48 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:29.221995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec4d53-e443-4305-ab7d-4ccd67a91c48 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:29.231398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 767/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:31.641282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f6f16-f536-4e89-902e-33c7089d860b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:31.641886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f6f16-f536-4e89-902e-33c7089d860b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:31.641886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f6f16-f536-4e89-902e-33c7089d860b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:31.643730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 755/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:31.649851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f272e-9bc1-49dc-a3ca-3de4b12e343a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:31.650290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f272e-9bc1-49dc-a3ca-3de4b12e343a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:31.650447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f272e-9bc1-49dc-a3ca-3de4b12e343a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:31.670277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-038f272e-9bc1-49dc-a3ca-3de4b12e343a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:31.671494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-038f272e-9bc1-49dc-a3ca-3de4b12e343a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:31.679148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 768/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:31.832555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f6f16-f536-4e89-902e-33c7089d860b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:31.847049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f6f16-f536-4e89-902e-33c7089d860b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:31.852489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8df6f5-3b25-4d1d-a807-f625708925df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:31.853084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8df6f5-3b25-4d1d-a807-f625708925df service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:31.853359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8df6f5-3b25-4d1d-a807-f625708925df service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:31.853630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8df6f5-3b25-4d1d-a807-f625708925df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:31.861609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8df6f5-3b25-4d1d-a807-f625708925df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:31.863195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8df6f5-3b25-4d1d-a807-f625708925df service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:31.872414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 756/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:32.944992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b330e-871c-475b-9565-2c348f36a7ec None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.945576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b330e-871c-475b-9565-2c348f36a7ec None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.945576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4b330e-871c-475b-9565-2c348f36a7ec None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.946878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 769/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 19:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:32.953083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307e66d-97eb-4935-867a-c530e95409c1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:32.953515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307e66d-97eb-4935-867a-c530e95409c1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:32.953652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307e66d-97eb-4935-867a-c530e95409c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:32.973681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8307e66d-97eb-4935-867a-c530e95409c1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:32.974908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8307e66d-97eb-4935-867a-c530e95409c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:32.982734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 757/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 19:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:33.132397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b330e-871c-475b-9565-2c348f36a7ec None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:33.146896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4b330e-871c-475b-9565-2c348f36a7ec None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:33.152245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4234c9-b75e-4343-9da2-e96aa9a9f32e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:33.152825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4234c9-b75e-4343-9da2-e96aa9a9f32e service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:33.153097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4234c9-b75e-4343-9da2-e96aa9a9f32e service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:33.153364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4234c9-b75e-4343-9da2-e96aa9a9f32e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:33.162125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4234c9-b75e-4343-9da2-e96aa9a9f32e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:33.163681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4234c9-b75e-4343-9da2-e96aa9a9f32e service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:33.172699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 770/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:37.315220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354a7b-82aa-4f1c-a9b8-49a61a3a4f3b None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:37.315771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354a7b-82aa-4f1c-a9b8-49a61a3a4f3b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:37.315771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86354a7b-82aa-4f1c-a9b8-49a61a3a4f3b None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:37.317079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 758/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:37.322187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a11b9f-03d8-482d-add9-862850c93dc3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:37.322622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a11b9f-03d8-482d-add9-862850c93dc3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:37.322760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a11b9f-03d8-482d-add9-862850c93dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:37.342611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-77a11b9f-03d8-482d-add9-862850c93dc3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:37.343809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-77a11b9f-03d8-482d-add9-862850c93dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:37.355265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 771/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:37.654698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354a7b-82aa-4f1c-a9b8-49a61a3a4f3b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:37.656420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-86354a7b-82aa-4f1c-a9b8-49a61a3a4f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:37.732995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86354a7b-82aa-4f1c-a9b8-49a61a3a4f3b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:37.736555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2daf3f-f664-45a2-b982-fa1d595e0006 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:37.737141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2daf3f-f664-45a2-b982-fa1d595e0006 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:37.737414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2daf3f-f664-45a2-b982-fa1d595e0006 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:37.737682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2daf3f-f664-45a2-b982-fa1d595e0006 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:37.746432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2daf3f-f664-45a2-b982-fa1d595e0006 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:37.747846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2daf3f-f664-45a2-b982-fa1d595e0006 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:37.757562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 759/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:39.893903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7e747-dc80-434b-9f01-37940fe89927 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:39.894438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7e747-dc80-434b-9f01-37940fe89927 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:39.894438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7e747-dc80-434b-9f01-37940fe89927 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:39.895811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 772/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:39.900916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299c21a-e253-4b3b-88c6-73af53d16943 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:39.901317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299c21a-e253-4b3b-88c6-73af53d16943 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:39.901459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e299c21a-e253-4b3b-88c6-73af53d16943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:39.921315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e299c21a-e253-4b3b-88c6-73af53d16943 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:39.922523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e299c21a-e253-4b3b-88c6-73af53d16943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:39.930759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 760/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:40.223469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7e747-dc80-434b-9f01-37940fe89927 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:40.234156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7e747-dc80-434b-9f01-37940fe89927 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:40.238380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffa149b-312f-4ec8-b22c-7d66ed4a53e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:40.238974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa149b-312f-4ec8-b22c-7d66ed4a53e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:40.239251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa149b-312f-4ec8-b22c-7d66ed4a53e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:40.239518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffa149b-312f-4ec8-b22c-7d66ed4a53e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:40.248338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa149b-312f-4ec8-b22c-7d66ed4a53e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:40.249760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffa149b-312f-4ec8-b22c-7d66ed4a53e0 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:40.258958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 773/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:43.961304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c4395-ed8a-437a-8bc5-bc05c1d70d17 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:43.961883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c4395-ed8a-437a-8bc5-bc05c1d70d17 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:43.961883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061c4395-ed8a-437a-8bc5-bc05c1d70d17 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:43.963495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 761/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:43.968516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594484a-a81e-4d31-add5-a537602660bd None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:43.968936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594484a-a81e-4d31-add5-a537602660bd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:43.969071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594484a-a81e-4d31-add5-a537602660bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:43.989062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5594484a-a81e-4d31-add5-a537602660bd None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:43.990398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5594484a-a81e-4d31-add5-a537602660bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:43.998980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 774/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:44.304000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c4395-ed8a-437a-8bc5-bc05c1d70d17 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:44.305762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-061c4395-ed8a-437a-8bc5-bc05c1d70d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:44.370097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:44.383779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061c4395-ed8a-437a-8bc5-bc05c1d70d17 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:44.387308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29a8e-f0c3-4551-8d68-576cc17af9a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:44.387904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29a8e-f0c3-4551-8d68-576cc17af9a9 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:44.388178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29a8e-f0c3-4551-8d68-576cc17af9a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:44.388446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29a8e-f0c3-4551-8d68-576cc17af9a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:44.397118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29a8e-f0c3-4551-8d68-576cc17af9a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:44.398590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29a8e-f0c3-4551-8d68-576cc17af9a9 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:44.407757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 762/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:48:46.549175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e968ec-be06-4cb3-856a-8ad108ed219c None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:46.549792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e968ec-be06-4cb3-856a-8ad108ed219c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:46.549792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e968ec-be06-4cb3-856a-8ad108ed219c None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:46.551084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 775/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 19:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:48:46.556350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e44400-6cab-445b-938b-90e0223e3717 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:46.556774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e44400-6cab-445b-938b-90e0223e3717 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:46.556920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e44400-6cab-445b-938b-90e0223e3717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:46.575891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-04e44400-6cab-445b-938b-90e0223e3717 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:48:46.577001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-04e44400-6cab-445b-938b-90e0223e3717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:46.585054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 763/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 19:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:48:46.885932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e968ec-be06-4cb3-856a-8ad108ed219c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:48:46.887600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-41e968ec-be06-4cb3-856a-8ad108ed219c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:48:46.901029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e968ec-be06-4cb3-856a-8ad108ed219c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:48:46.904553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d19005-7c0b-410c-aebf-c5d9f737c954 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:48:46.905169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d19005-7c0b-410c-aebf-c5d9f737c954 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:48:46.905448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d19005-7c0b-410c-aebf-c5d9f737c954 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:48:46.905721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d19005-7c0b-410c-aebf-c5d9f737c954 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:48:46.914765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d19005-7c0b-410c-aebf-c5d9f737c954 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:48:46.916170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d19005-7c0b-410c-aebf-c5d9f737c954 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:48:46.925471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 776/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:48:56.571019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=11 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:48:56.921333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:13.300550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.301204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.301204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.334483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.335708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.350155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 764/1540] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:13.359980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d382a-e6b4-4048-ae17-3eedda866098 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.360910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d382a-e6b4-4048-ae17-3eedda866098 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.361347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83d382a-e6b4-4048-ae17-3eedda866098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.392935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f83d382a-e6b4-4048-ae17-3eedda866098 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.394617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f83d382a-e6b4-4048-ae17-3eedda866098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.410193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 777/1541] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.415125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.428909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a83753-70c9-40cc-959d-db6b2ab38546 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.433973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.434535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.436575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.436575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.436575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:13.436575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.437654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.456983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 765/1542] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:13.463056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041b590-514c-4409-aff3-835ab433b361 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.463594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041b590-514c-4409-aff3-835ab433b361 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.464039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8041b590-514c-4409-aff3-835ab433b361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.465119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.476080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831206ee-2f98-48a8-8758-279a718d4c01 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.491142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.491759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.492033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.492305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.492946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.493991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:13.493991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:50:13.493991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:50:13.497957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60146ca-4bef-4bbf-a603-d0352273fcb5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.503817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8041b590-514c-4409-aff3-835ab433b361 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.505257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8041b590-514c-4409-aff3-835ab433b361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.525870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 778/1543] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.545098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.546050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.546652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.561855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 766/1544] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:13.563151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89c54b-6da1-40e1-97cf-56338cd50f8c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.564201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89c54b-6da1-40e1-97cf-56338cd50f8c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.564201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89c54b-6da1-40e1-97cf-56338cd50f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.586339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.588163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.602803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2b89c54b-6da1-40e1-97cf-56338cd50f8c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.604029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2b89c54b-6da1-40e1-97cf-56338cd50f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.681862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 779/1545] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:13.683015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.689113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 767/1546] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.690502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590eaed-0b36-4459-899a-b397d7ec178b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.690888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590eaed-0b36-4459-899a-b397d7ec178b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.691256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7590eaed-0b36-4459-899a-b397d7ec178b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.694696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b679ca-c8fc-4cd7-b0b2-24153e9544de None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.699809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.700407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.701338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.701609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.702469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.704537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:13.704537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:50:13.704537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:50:13.705745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70666923-07db-46bc-b0ad-20b261572489 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.714177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7590eaed-0b36-4459-899a-b397d7ec178b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.715495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7590eaed-0b36-4459-899a-b397d7ec178b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.725574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 768/1547] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.726880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8dcd-a934-48d1-b459-e8541bc628da None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.727284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8dcd-a934-48d1-b459-e8541bc628da None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.727559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27f8dcd-a934-48d1-b459-e8541bc628da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.733168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 780/1548] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:13.734518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.734789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.735058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.751652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f27f8dcd-a934-48d1-b459-e8541bc628da None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.752941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f27f8dcd-a934-48d1-b459-e8541bc628da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.757272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.758608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.761282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 769/1549] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.762369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a294d-43c1-4945-91f3-528e97a09eb4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.762752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a294d-43c1-4945-91f3-528e97a09eb4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.763678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a294d-43c1-4945-91f3-528e97a09eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.769560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 781/1550] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.770681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.780764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddcad23-914d-4bf4-822d-58c3097107d4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.785694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.786322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.786602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.786875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.787517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:13.787821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.788833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:13.788833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:50:13.788833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:50:13.789709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650948b2-7e73-45ba-82ce-3f5d3173235f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.792058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d72a294d-43c1-4945-91f3-528e97a09eb4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.793480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d72a294d-43c1-4945-91f3-528e97a09eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.801657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 782/1551] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:13.802003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 770/1552] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.803424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.803936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.804435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.808630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5be48-3eb8-43f5-9c43-80020d224343 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.809165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5be48-3eb8-43f5-9c43-80020d224343 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.809450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5be48-3eb8-43f5-9c43-80020d224343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.834157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.834387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c4c5be48-3eb8-43f5-9c43-80020d224343 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.835457 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c4c5be48-3eb8-43f5-9c43-80020d224343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.835579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.843589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 771/1553] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.843728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 783/1554] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.847391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.847574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed29266-668e-4731-86ee-ea76106f559a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.847574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed29266-668e-4731-86ee-ea76106f559a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.847574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed29266-668e-4731-86ee-ea76106f559a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.856689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4093466-3baa-49e3-a445-720d560fa037 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.861270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4c33f6-438f-4bd5-bf0b-d2eaf2b70e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.861885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c33f6-438f-4bd5-bf0b-d2eaf2b70e9a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.862164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c33f6-438f-4bd5-bf0b-d2eaf2b70e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.862427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c33f6-438f-4bd5-bf0b-d2eaf2b70e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.863055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c33f6-438f-4bd5-bf0b-d2eaf2b70e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.864627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4c33f6-438f-4bd5-bf0b-d2eaf2b70e9a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.864782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5ed29266-668e-4731-86ee-ea76106f559a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.866071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5ed29266-668e-4731-86ee-ea76106f559a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.874454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 784/1555] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.877763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 772/1556] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:13.877891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.877891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.878058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88855386-e54e-46a6-ae3c-26f3c758b2a5 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.878162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.878253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88855386-e54e-46a6-ae3c-26f3c758b2a5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.878253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88855386-e54e-46a6-ae3c-26f3c758b2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.895769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.896310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-88855386-e54e-46a6-ae3c-26f3c758b2a5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.896964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.897458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-88855386-e54e-46a6-ae3c-26f3c758b2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.904753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 785/1557] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.906015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 773/1558] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.907279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.907651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.908270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.909262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.918823 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfaefd-d5e9-4159-b649-d3b829076b5b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.923874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.924467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.924777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.925049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.925670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.927590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.927950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:13.929120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:13.937820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 774/1559] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:13.938584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.947965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d986a38-2b22-40e9-91d2-defd8f39beaa None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:13.954568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:13.955158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:13.955428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:13.955688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:13.956321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:13.956616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:13.958222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:13.963271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 786/1560] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:13.963271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.977678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 775/1561] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:13.977678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:13.981159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cc61b-3311-4038-abdd-6e45351fbf6c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.024520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.030670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c5f0a7-eb7c-4232-b0a3-2644abd93e39 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.035918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.036526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.036807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.037081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.037761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:14.038052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.039651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.040181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.040977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.041250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.041517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.042187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.043784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.051055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 787/1562] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:14.053279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 776/1563] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.058664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.058809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.064183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.069293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3fa7b-7ef5-4bf4-a801-ed1873de13a3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.070282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a0fea-88a3-4c3d-b824-d2ad2b7e9277 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.074240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.074826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.075106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.075244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.075368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.076361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.076553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:14.076666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.076856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.076951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.077508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:14.077843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.078461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57694a79-62aa-4741-ad99-33e457a1dca3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.079414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.092083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 788/1564] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:14.092542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 777/1565] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:14.092874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.094576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.094857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.095123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.108357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce76f12f-42de-4a49-90cb-3d163af629ce admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.112894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.113480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.113784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.114056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.114685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:14.114983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.115882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:14.116562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.117065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.128281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 778/1566] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:14.128774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 789/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:14.129216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.129538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.138715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1257fe5-f828-4aef-ab68-3cf8ad161857 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.139558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb8a20e-f05d-4182-a556-d08c0952bfab admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.143264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.143840 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.144133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.144263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.144428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.144715 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.144993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.145094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.145260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.145901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.146919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.147769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.170231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 779/1568] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.171007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.175872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 790/1569] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 393 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.176619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.184068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf0a10-28e7-4089-9b4a-a181a4fcd99a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.243268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.244235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.244650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.245055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] PATH_INFO: `/v3/projects/ccf257bf6ccd476b839dbe55c8b27d4b/users/0792c6c88f9c4534aad5bf7ea3a4ac60/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.245835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391d3b61-63b1-4517-89d9-20ac8ef449d4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.251177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.251774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.252048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.252314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.252932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.254912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.268053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf257bf6ccd476b839dbe55c8b27d4b, user_id=0792c6c88f9c4534aad5bf7ea3a4ac60, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.269803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:14.269803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:50:14.269803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:50:14.272105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.284106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 791/1570] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.285088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.288509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 780/1571] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:14 2022] PUT /identity/v3/projects/ccf257bf6ccd476b839dbe55c8b27d4b/users/0792c6c88f9c4534aad5bf7ea3a4ac60/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:14.289649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.348944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655ad366-3fa5-4ab1-8fa1-b471ebee40fc admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.353644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.354278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.354668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.354948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.355601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.356192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe5464-bec3-4876-b144-55c1b1727968 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.358094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.361299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.361940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.362219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.362655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.363611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.365497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.387942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 792/1572] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.389099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.398743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 781/1573] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.398743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.409928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.418106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.457761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2c632-c3d4-42c2-bd89-aa1deeb0f0b4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.462332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.462928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.463205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.463478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.464107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:14.464405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.465902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d1126a-aa72-4aff-9de3-9c39a049b5e8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.466547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.471247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.471843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.472120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.472417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.473069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.475000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.478398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 793/1574] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:14.479375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.489476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b0081-8656-424d-a6d1-bb933d5d411e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.494103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.494614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.494892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.495174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.495784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.497428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.506281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 782/1575] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.506281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.509896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 794/1576] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.510716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.521168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70412440-0a73-4fde-b3b7-c332ccd27411 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.566737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c787f1-d414-48c3-9441-d48e5a9439ab admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.571410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3ee89c-390b-4ff1-a387-f8796953677f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.572010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee89c-390b-4ff1-a387-f8796953677f admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.572282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee89c-390b-4ff1-a387-f8796953677f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.572548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3ee89c-390b-4ff1-a387-f8796953677f admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.573182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ee89c-390b-4ff1-a387-f8796953677f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.575093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3ee89c-390b-4ff1-a387-f8796953677f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.579762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.580365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.580641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.580909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.581607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.583632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.598081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 783/1577] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.599337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.599620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.599895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.609557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 795/1578] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.609557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.622498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58775466-7a16-43ed-8808-00b523d79d13 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.627348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.627935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.628206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.628402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:14.628636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.629087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.629430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:14.631105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.638407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 784/1579] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:14.639182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.649080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5a3f3f-6f08-4c60-9d23-84165e760d9c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.654916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.655517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.655791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.656056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.656734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.658911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.659714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 796/1580] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 293 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.660685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.671599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9592461d-e475-4794-9818-24645640d4a1 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.677500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.678201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.678482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.678756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.679378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.681392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.682926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 785/1581] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 291 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.683735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.693177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f910aed-433e-4711-b29d-f29d9ad84867 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.697695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.698308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.698582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.698849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.699456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.701427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.711176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 797/1582] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.714894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.724969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 786/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 284 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.725816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.737602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9626e3d-9397-485c-8070-4197d8fdca4e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.774874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.779129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030c706-f0d8-4ea8-9ae3-77270b44ffe3 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.781146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:14.785169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.785950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.786151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.786406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.787013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.788723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.796638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.797243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.797524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.797825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.798470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.799243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 798/1584] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.800040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.803182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.809414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa9ce6b-59ba-4f85-883c-9e16f94b2510 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.816185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 787/1585] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.816185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.816408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.816408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.816408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.816408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.816742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.817300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.828592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 799/1586] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.829425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.842642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84056a40-22b4-461e-88c0-6e5027ace907 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.847393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.847984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.848267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.848537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.849169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.850787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.860980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 800/1587] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.860980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.870993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09e0547-2a14-490a-b8d7-1c2f75f1c11d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.875731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.876449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.876577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042212e-7625-49e5-b0d6-7e6c498343eb admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.876731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.877040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.878397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.880220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.884488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.884948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.885135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.885317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.892212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 801/1588] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.896356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.896989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.897911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.907604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8875c54-b161-44fc-8c02-da72e8672ea9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.908830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 788/1589] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:14.909517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.913154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.913873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.914421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.914769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.916141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.918468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.919782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9767c0d3-732f-496d-94ed-47c6016adecd service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.925164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.925796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.926077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.926381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.927044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.929280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.941798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 789/1590] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.942620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.944371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 802/1591] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:14 2022] POST /identity/v3/users => generated 288 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:14.945414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.957088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474685d8-6fa8-4f59-8d73-2fd75459308c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.959302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033aa9b-788e-4156-b317-f7875e389a4c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.962163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.962901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.963284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.963653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.964393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.964594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.965426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.965712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.966107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] PATH_INFO: `/v3/projects/2c9b3a5000944b779dfa88eb1166a162/users/edfc4ed9b9e844cd9073c15f9efad6db/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:14.967160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.983664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 803/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:14.983664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:14.984140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c9b3a5000944b779dfa88eb1166a162, user_id=edfc4ed9b9e844cd9073c15f9efad6db, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:14.987072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:14.992807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6e128f-e986-4b6d-b92a-499308919b91 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:14.998630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:14.999235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:14.999527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:14.999791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.000437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.002549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 790/1593] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:14 2022] PUT /identity/v3/projects/2c9b3a5000944b779dfa88eb1166a162/users/edfc4ed9b9e844cd9073c15f9efad6db/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.003629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.010179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.025447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 804/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:14 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.026539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.045752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92d7ccf-6877-4d86-a687-1d23caeca8a4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.099629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba704d50-7624-4a36-ba6f-f109a09bcdcb admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.105004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.105685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.106065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.106415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.107175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.109043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.111406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.113027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.114866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.115324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.116113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.119893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.123433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 791/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.124371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.135323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529222e-98b5-4a2d-8087-4a2e827d62b4 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.142127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 805/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.142127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.142318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.142422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.142707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.142976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.143614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.145227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.158185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 792/1597] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.162237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.165052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfea3e0-e7fc-43b9-9702-5ad02a6b3f83 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.170818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f26214-a19f-4726-93ef-25e8c0b397b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.171522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26214-a19f-4726-93ef-25e8c0b397b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.171810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26214-a19f-4726-93ef-25e8c0b397b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.172085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f26214-a19f-4726-93ef-25e8c0b397b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.173385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26214-a19f-4726-93ef-25e8c0b397b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.175098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f26214-a19f-4726-93ef-25e8c0b397b1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.176029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7b574-8584-4f50-ac00-25a6cf24f9bf admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.182306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.183195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.183694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.184134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] PATH_INFO: `/v3/projects/f82690ff3ee94e708caf1d4f7d1979c2/users/53fa55dda30e4aaa86ae599e4df2fed3/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.189196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 806/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.190480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.190767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.191041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.203232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f82690ff3ee94e708caf1d4f7d1979c2, user_id=53fa55dda30e4aaa86ae599e4df2fed3, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.206217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.221978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] MFA Rules not processed for user `edfc4ed9b9e844cd9073c15f9efad6db`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.223206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.224145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 793/1599] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/f82690ff3ee94e708caf1d4f7d1979c2/users/53fa55dda30e4aaa86ae599e4df2fed3/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.225162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.240357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.298946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.304145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48dcff9-68da-4700-8fbe-070815bda44d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.310486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.311145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.311333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 807/1600] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.316650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.316650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] PATH_INFO: `/v3/projects/7d4531a73035420fb3f73e1fd43af258/users/eff856b09e5944a9bdcf7374cd9d4a2a/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.316838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.331933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754e2d8-936a-4eb0-b01e-fdf01b00ed9c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.334874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d4531a73035420fb3f73e1fd43af258, user_id=eff856b09e5944a9bdcf7374cd9d4a2a, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.338536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.344345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.345270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.345860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.348215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] PATH_INFO: `/v3/projects/31a694d77d3c4c21bccde9920a14dc88/users/6744d5ca67184164bacc6eb19dae000f/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.353669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 794/1601] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/7d4531a73035420fb3f73e1fd43af258/users/eff856b09e5944a9bdcf7374cd9d4a2a/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.354350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.374420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a694d77d3c4c21bccde9920a14dc88, user_id=6744d5ca67184164bacc6eb19dae000f, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.378663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.400951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 808/1602] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/31a694d77d3c4c21bccde9920a14dc88/users/6744d5ca67184164bacc6eb19dae000f/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.401862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.452145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530d46-20ec-4809-b9d4-c37c74fd7833 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.487629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda8c588-67c2-49d1-8900-4bb7655c2191 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.492918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.493583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.493902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.494171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] PATH_INFO: `/v3/projects/12bfdb3ea95a4cefa8f3c01be875ef14/users/894ca5b657fe4483935459ef7989244f/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.511278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12bfdb3ea95a4cefa8f3c01be875ef14, user_id=894ca5b657fe4483935459ef7989244f, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.515239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.518469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8499d4-c48d-454c-b236-8cc4eb97dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.519137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8499d4-c48d-454c-b236-8cc4eb97dd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.519470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8499d4-c48d-454c-b236-8cc4eb97dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.519749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8499d4-c48d-454c-b236-8cc4eb97dd4f admin admin] PATH_INFO: `/v3/projects/77649d227cf34946a5a47a86c1c3e7ef/users/4c46716661414f74b3ed33c302d116df/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.533688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 809/1603] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/12bfdb3ea95a4cefa8f3c01be875ef14/users/894ca5b657fe4483935459ef7989244f/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.535377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.539971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8499d4-c48d-454c-b236-8cc4eb97dd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=77649d227cf34946a5a47a86c1c3e7ef, user_id=4c46716661414f74b3ed33c302d116df, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.542931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8499d4-c48d-454c-b236-8cc4eb97dd4f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.561782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 795/1604] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:15 2022] PUT /identity/v3/projects/77649d227cf34946a5a47a86c1c3e7ef/users/4c46716661414f74b3ed33c302d116df/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:15.561782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae07a-f4f0-4438-98e4-c58d167d65fb None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.561782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae07a-f4f0-4438-98e4-c58d167d65fb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.561782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cae07a-f4f0-4438-98e4-c58d167d65fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.580142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-09cae07a-f4f0-4438-98e4-c58d167d65fb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.581347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-09cae07a-f4f0-4438-98e4-c58d167d65fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.600831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.631317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a5f814-5e8e-4e14-82d4-e01efc1e9cf9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.651666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 796/1605] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.651666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2453aed-64cc-4269-a3c1-aa87fa1c104f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.651666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2453aed-64cc-4269-a3c1-aa87fa1c104f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.651666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2453aed-64cc-4269-a3c1-aa87fa1c104f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.670585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f2453aed-64cc-4269-a3c1-aa87fa1c104f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.671926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f2453aed-64cc-4269-a3c1-aa87fa1c104f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.679883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 797/1606] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.683714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7aa2-3e74-431f-9c90-234da8836500 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.683960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7aa2-3e74-431f-9c90-234da8836500 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.684352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd7aa2-3e74-431f-9c90-234da8836500 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.686703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 798/1607] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 19:50:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:15.687824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497833-04b1-446d-a59f-7d079b367d72 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.688105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497833-04b1-446d-a59f-7d079b367d72 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.688500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48497833-04b1-446d-a59f-7d079b367d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.711695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34795af4-d392-402a-a695-698672abe43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.711695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795af4-d392-402a-a695-698672abe43d admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.711901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795af4-d392-402a-a695-698672abe43d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.712040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34795af4-d392-402a-a695-698672abe43d admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.712695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795af4-d392-402a-a695-698672abe43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.714411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34795af4-d392-402a-a695-698672abe43d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.717076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:15.725649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-48497833-04b1-446d-a59f-7d079b367d72 None None] MFA Rules not processed for user `6744d5ca67184164bacc6eb19dae000f`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.727219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-48497833-04b1-446d-a59f-7d079b367d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.729453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 810/1608] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:15.731045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.731335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.732171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.767382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] MFA Rules not processed for user `894ca5b657fe4483935459ef7989244f`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.769027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.815387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 799/1609] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.816766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736330-182c-4f03-9eed-d34369c5a686 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.817051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736330-182c-4f03-9eed-d34369c5a686 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.817338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b736330-182c-4f03-9eed-d34369c5a686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.859726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3b736330-182c-4f03-9eed-d34369c5a686 None None] MFA Rules not processed for user `4c46716661414f74b3ed33c302d116df`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:15.863993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3b736330-182c-4f03-9eed-d34369c5a686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:15.878042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 811/1610] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.879191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:15.956306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f46457-7c32-4543-ae60-256ad6e3bee1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:15.961542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:15.962257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.962604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.962949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.963206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 800/1611] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:15.964509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:15.964855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:15.965133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:15.978620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:15.980858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:15.994995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 812/1612] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:15.995948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.006841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47effff0-8a80-414d-8218-8c4c05ab36d5 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.012027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.012371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.012955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.013331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.013915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.014217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.025549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.027528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.041887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 813/1613] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:16.043945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.044182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.055271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d763ed3-94ac-4b5d-830a-5c2fa2e8ff70 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.062567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c26f2-8b1d-4eb3-9b29-ea9ca5499d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.063180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c26f2-8b1d-4eb3-9b29-ea9ca5499d18 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.063457 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c26f2-8b1d-4eb3-9b29-ea9ca5499d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.063733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c26f2-8b1d-4eb3-9b29-ea9ca5499d18 admin admin] PATH_INFO: `/v3/projects/b6912d8a0f544d9d9b2d9b1d1048952d/users/91a5130420764ac78383961d7665a964/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.082889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c26f2-8b1d-4eb3-9b29-ea9ca5499d18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6912d8a0f544d9d9b2d9b1d1048952d, user_id=91a5130420764ac78383961d7665a964, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.086835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c26f2-8b1d-4eb3-9b29-ea9ca5499d18 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.109805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 814/1614] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:16 2022] PUT /identity/v3/projects/b6912d8a0f544d9d9b2d9b1d1048952d/users/91a5130420764ac78383961d7665a964/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:16.111622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c15820-f485-453f-bd13-d6971f08a638 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.111908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c15820-f485-453f-bd13-d6971f08a638 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.112286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c15820-f485-453f-bd13-d6971f08a638 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.113773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 815/1615] 10.222.0.15 () {54 vars in 962 bytes} [Tue Sep 13 19:50:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:16.114291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c15820-f485-453f-bd13-d6971f08a638 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.120602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 801/1616] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 19:50:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:16.128543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.139400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2230e2-21d5-4f60-a655-98869f1d4da4 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.195977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c15820-f485-453f-bd13-d6971f08a638 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.201038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b9c7f5-6717-4464-af8c-bcb91e581dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.201790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c7f5-6717-4464-af8c-bcb91e581dcc service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.202411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c7f5-6717-4464-af8c-bcb91e581dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.202546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b9c7f5-6717-4464-af8c-bcb91e581dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.204429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7ee24e-6256-49fd-a96b-8563fe0861ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.205040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ee24e-6256-49fd-a96b-8563fe0861ae admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.205664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ee24e-6256-49fd-a96b-8563fe0861ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.206065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7ee24e-6256-49fd-a96b-8563fe0861ae admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.206712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ee24e-6256-49fd-a96b-8563fe0861ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.209295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7ee24e-6256-49fd-a96b-8563fe0861ae admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.212249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.243564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 802/1617] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/projects => generated 381 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:16.245708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.246434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.246864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.273370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.275521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.280993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9c7f5-6717-4464-af8c-bcb91e581dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.282534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b9c7f5-6717-4464-af8c-bcb91e581dcc service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.293533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 816/1618] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:16.295464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.295753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.296037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.317865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:16.319194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.362000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 803/1619] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:16.366170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.435637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:16.444925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 817/1620] 10.222.0.15 () {58 vars in 1059 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:16.446598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.474815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ca159-38d8-431c-930a-8d563d1ca9bb None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.476834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb67e9-a541-4e39-9514-00a50a76ffc1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.482868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.483377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.483694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.484025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.487634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.489405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.489789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.490072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.491287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.496004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.531645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 818/1621] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:16 2022] POST /identity/v3/users => generated 297 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:16.533900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.541023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:16.578734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.580699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.593647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 804/1622] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:16.594700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.610776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82c810-fb0b-4f2a-8904-c9b3743c5ab9 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.617537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.618444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.618602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.618785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.623851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b96158-0711-4a4e-8f0e-36a30b29dbc0 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.628778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.630426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.630709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.630986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.644819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.647473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.676143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 819/1623] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:16.680249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.723072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766181f-b9c0-4105-8aea-46195c77bfe3 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.725411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.730847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.737366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.738258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.738541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.738816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.739468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.741358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.762209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 805/1624] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:16.769233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.771207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 820/1625] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:16.789841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749c82b-5c9d-4dc9-a5d2-9287582e3554 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.797883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:16.798624 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:16.798902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:16.799268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] PATH_INFO: `/v3/projects/6991de03924d4af2a160e795cad82e5b/users/3ff93ca22f8e478596d3bca6a8cf3abd/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:16.829546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6991de03924d4af2a160e795cad82e5b, user_id=3ff93ca22f8e478596d3bca6a8cf3abd, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:16.841590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:16.875325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 806/1626] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:16 2022] PUT /identity/v3/projects/6991de03924d4af2a160e795cad82e5b/users/3ff93ca22f8e478596d3bca6a8cf3abd/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:16.875735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:16.895327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d870a25-447e-4930-9a69-e829097f6cdc admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:16.944958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:17.177708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64053e60-42bb-43d5-89e8-69a3b873709c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.177708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64053e60-42bb-43d5-89e8-69a3b873709c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.177708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64053e60-42bb-43d5-89e8-69a3b873709c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.177708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64053e60-42bb-43d5-89e8-69a3b873709c admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.183867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64053e60-42bb-43d5-89e8-69a3b873709c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.183867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64053e60-42bb-43d5-89e8-69a3b873709c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.183867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 821/1627] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:16 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:17.185615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.187086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a96b5ca-c4e8-4c41-a99a-3bdc891a6a72 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.196524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409bd94-aa2f-45a7-8125-9a18440f26fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.198043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409bd94-aa2f-45a7-8125-9a18440f26fe admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.198043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409bd94-aa2f-45a7-8125-9a18440f26fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.199177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409bd94-aa2f-45a7-8125-9a18440f26fe admin admin] PATH_INFO: `/v3/projects/6991de03924d4af2a160e795cad82e5b/users/3ff93ca22f8e478596d3bca6a8cf3abd/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.201399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409bd94-aa2f-45a7-8125-9a18440f26fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=6991de03924d4af2a160e795cad82e5b, user_id=3ff93ca22f8e478596d3bca6a8cf3abd, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.204595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409bd94-aa2f-45a7-8125-9a18440f26fe admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.236730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 807/1628] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:17 2022] PUT /identity/v3/projects/6991de03924d4af2a160e795cad82e5b/users/3ff93ca22f8e478596d3bca6a8cf3abd/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:17.240448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd110566-3edb-49d3-b70e-1780cba30cac None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.240743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd110566-3edb-49d3-b70e-1780cba30cac None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.241002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd110566-3edb-49d3-b70e-1780cba30cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.322815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6c40-42e4-4234-9c7a-9237c7b85bad None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.323237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6c40-42e4-4234-9c7a-9237c7b85bad None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.323605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d6c40-42e4-4234-9c7a-9237c7b85bad None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.325370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 808/1629] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 19:50:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:17.328440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-dd110566-3edb-49d3-b70e-1780cba30cac None None] MFA Rules not processed for user `3ff93ca22f8e478596d3bca6a8cf3abd`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:17.331142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.331467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.331839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.335555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-dd110566-3edb-49d3-b70e-1780cba30cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.365084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:17.366463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.451306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:17.521201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 822/1630] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:17.530688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 809/1631] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:17.531422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de78484-2fe8-4fdf-a3f5-115918cb5b70 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.531880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de78484-2fe8-4fdf-a3f5-115918cb5b70 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.532298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de78484-2fe8-4fdf-a3f5-115918cb5b70 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.534300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 823/1632] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 19:50:17 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:17.539757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.544773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de78484-2fe8-4fdf-a3f5-115918cb5b70 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.633029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080ed2c-870a-4f06-a355-ef904edf24f1 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.637865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745ee33-dfee-4f12-b3a8-6c33bd12ca21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.638559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ee33-dfee-4f12-b3a8-6c33bd12ca21 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.638899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ee33-dfee-4f12-b3a8-6c33bd12ca21 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.639281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745ee33-dfee-4f12-b3a8-6c33bd12ca21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.654104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745ee33-dfee-4f12-b3a8-6c33bd12ca21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.655692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745ee33-dfee-4f12-b3a8-6c33bd12ca21 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.656883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de78484-2fe8-4fdf-a3f5-115918cb5b70 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.664206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.665469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.665988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.666357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.669556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 810/1633] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:17.671028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.671313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.671589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.679489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.681250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.700321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 824/1634] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:17.708169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:17.709717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:17.730450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 811/1635] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:17.778951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:17.788921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda4eb28-8292-4e4a-947e-38721fce3f46 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:17.801652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:17.802456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:17.802831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:17.803200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:17.827812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:17.829411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:17.860640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 825/1636] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:18.721190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:18.731291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36520d96-a22d-4e6c-8875-74a5acad94f0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:18.736554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda95c6-408a-4273-a8e8-d4f392b65df2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:18.737492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda95c6-408a-4273-a8e8-d4f392b65df2 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:18.737961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda95c6-408a-4273-a8e8-d4f392b65df2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:18.738400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda95c6-408a-4273-a8e8-d4f392b65df2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:18.817905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda95c6-408a-4273-a8e8-d4f392b65df2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:18.819587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda95c6-408a-4273-a8e8-d4f392b65df2 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:18.845880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 812/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:18 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:22.063211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:22.117054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:22.219366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df731016-b801-4833-8e75-4379ee9244f8 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:22.238531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:22.239853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.240140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.241576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] PATH_INFO: `/v3/users/edfc4ed9b9e844cd9073c15f9efad6db` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.244166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfc4ed9b9e844cd9073c15f9efad6db)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:22.253695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:22.371873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfc4ed9b9e844cd9073c15f9efad6db'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.393456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfc4ed9b9e844cd9073c15f9efad6db'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.394159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfc4ed9b9e844cd9073c15f9efad6db'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.395952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfc4ed9b9e844cd9073c15f9efad6db'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.411651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfc4ed9b9e844cd9073c15f9efad6db'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:22.448148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 826/1638] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:22 2022] DELETE /identity/v3/users/edfc4ed9b9e844cd9073c15f9efad6db => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:22.451379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b5521-9325-4642-b050-806a964887f7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.451946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b5521-9325-4642-b050-806a964887f7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.452024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23b5521-9325-4642-b050-806a964887f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.490270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f23b5521-9325-4642-b050-806a964887f7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:22.492048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f23b5521-9325-4642-b050-806a964887f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:22.601974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 813/1639] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:22.756913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:22.758703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:22.922935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240dabc2-0859-4587-bf35-325a433f03e6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:22.938032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d23cecc-3514-400a-886d-10d3c9ab7436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:22.938665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cecc-3514-400a-886d-10d3c9ab7436 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:22.945979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cecc-3514-400a-886d-10d3c9ab7436 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:22.946378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d23cecc-3514-400a-886d-10d3c9ab7436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:22.970262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cecc-3514-400a-886d-10d3c9ab7436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:22.971729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d23cecc-3514-400a-886d-10d3c9ab7436 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:23.001973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 827/1640] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:24.298127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b5521-9325-4642-b050-806a964887f7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.300461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f23b5521-9325-4642-b050-806a964887f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.328683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23b5521-9325-4642-b050-806a964887f7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.348496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.349171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.349448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.349959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] PATH_INFO: `/v3/projects/2c9b3a5000944b779dfa88eb1166a162` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.351848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c9b3a5000944b779dfa88eb1166a162)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.354197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:24.358756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:24.546091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c9b3a5000944b779dfa88eb1166a162', 'request_id': 'req-3e9a3474-50aa-4858-a08a-284fde38585f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.554351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c9b3a5000944b779dfa88eb1166a162', 'request_id': 'req-3e9a3474-50aa-4858-a08a-284fde38585f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.582461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c9b3a5000944b779dfa88eb1166a162', 'request_id': 'req-3e9a3474-50aa-4858-a08a-284fde38585f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.633628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] The token cache is being invalidate because project 2c9b3a5000944b779dfa88eb1166a162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:24.634189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.635019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3e9a3474-50aa-4858-a08a-284fde38585f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:24.637209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 814/1641] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:24 2022] DELETE /identity/v3/projects/2c9b3a5000944b779dfa88eb1166a162 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:24.643258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310432-bb6b-45a2-9d3e-6845b21ab29c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.643802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310432-bb6b-45a2-9d3e-6845b21ab29c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.643947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78310432-bb6b-45a2-9d3e-6845b21ab29c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.671175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.671551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.671898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.686611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-78310432-bb6b-45a2-9d3e-6845b21ab29c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:24.691029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-78310432-bb6b-45a2-9d3e-6845b21ab29c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.732645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] MFA Rules not processed for user `91a5130420764ac78383961d7665a964`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:24.734516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.814010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:24.851928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 828/1642] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:24.854177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.854461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.854741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.866761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 815/1643] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:24.877038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.878652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.896895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:24.904502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:24.952830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 829/1644] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:24.954930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:24.969846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6d3cc-3e25-4532-9593-17e62966446c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:24.977831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:24.978476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:24.978839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:24.979135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:24.979801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:24.980078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:24.982902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.023137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 830/1645] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:25.029001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.059188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c119f-9204-4476-91c1-daae687effa5 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.061651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f873bec-1f56-4de1-9709-14ebb257f7bc None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.068991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.069626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.070291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.070578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.071231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.075709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.076338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.076617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.076941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.078171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.093661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.095288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.114430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 816/1646] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:25.131214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 831/1647] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:25.135968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.148389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:25.155997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d60615-f27e-46ca-bac6-27c88b48fbfb service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.275222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.276147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.276879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.277154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.279083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.281325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.317364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 817/1648] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/users => generated 287 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:25.320970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.333753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197b624-71a0-4c93-93f3-73645341ec7c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.339630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.341352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.341693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.342096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.342713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.344620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.359144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 832/1649] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:25.362894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.377002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fca98-f4f7-43ec-ab5e-2a23f53c2f60 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.384174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ec347f-f7c9-4ede-940e-814bd2930048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.385483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec347f-f7c9-4ede-940e-814bd2930048 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.386076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec347f-f7c9-4ede-940e-814bd2930048 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.386621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ec347f-f7c9-4ede-940e-814bd2930048 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.388054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec347f-f7c9-4ede-940e-814bd2930048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.390547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ec347f-f7c9-4ede-940e-814bd2930048 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.414298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 818/1650] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:25.418856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.446007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce241d0-bebc-498c-af6a-7ba500dc6b05 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.460063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.463152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.463989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.463989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] PATH_INFO: `/v3/projects/407a94a20e5944078ba583a23c93a80d/users/36fe4ddb2cf842e1b37bfa9f5210ff8b/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.510556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=407a94a20e5944078ba583a23c93a80d, user_id=36fe4ddb2cf842e1b37bfa9f5210ff8b, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.515662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.538643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 833/1651] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:25 2022] PUT /identity/v3/projects/407a94a20e5944078ba583a23c93a80d/users/36fe4ddb2cf842e1b37bfa9f5210ff8b/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:25.550268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.550707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.551077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.583762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] MFA Rules not processed for user `36fe4ddb2cf842e1b37bfa9f5210ff8b`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:25.585207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:25.698262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 819/1652] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:25.705031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.796762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cf19b-2b4b-4f65-a634-55d4571c7907 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:25.802107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:25.803046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:25.803433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:25.803810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:25.804585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:25.806819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:25.863035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:25.864486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 834/1653] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:25.886117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a17660-45c9-4b51-ad4c-6b4f3c07166c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.006991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.008419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.008887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.009332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.010183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.013232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.067466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 820/1654] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:25 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:26.072045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.089444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea9cb32-4f54-466b-a7df-0ab78a6fed12 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.101597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.103010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.103010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.103185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.103888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.106855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.123470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 835/1655] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:26.128327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.134141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.145525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03e3f50-1525-42ef-8240-4d37ae19a562 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.150788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.152733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.152733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.153196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] PATH_INFO: `/v3/projects/5c3cf3e1edc8443cb8bb2257f61a98f6/users/e4c0d774e24f49c680eb85fa5132e2d5/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.178066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3cf3e1edc8443cb8bb2257f61a98f6, user_id=e4c0d774e24f49c680eb85fa5132e2d5, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.183347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.210665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 821/1656] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:26 2022] PUT /identity/v3/projects/5c3cf3e1edc8443cb8bb2257f61a98f6/users/e4c0d774e24f49c680eb85fa5132e2d5/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:26.211947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.224782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98942d6f-b514-48d7-bd36-d0480c6faa60 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.332506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:26.367511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68023be7-ce2b-44a4-824e-78b09d1e90bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.369367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023be7-ce2b-44a4-824e-78b09d1e90bd admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.369606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023be7-ce2b-44a4-824e-78b09d1e90bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.370498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68023be7-ce2b-44a4-824e-78b09d1e90bd admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.372534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023be7-ce2b-44a4-824e-78b09d1e90bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.377399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68023be7-ce2b-44a4-824e-78b09d1e90bd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.396085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 836/1657] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:26 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:26.396770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.413852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aad7df6-62d3-4d2c-8155-247210773a0b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.422375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.423269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.423738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.424184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] PATH_INFO: `/v3/projects/5c3cf3e1edc8443cb8bb2257f61a98f6/users/e4c0d774e24f49c680eb85fa5132e2d5/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.428547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c3cf3e1edc8443cb8bb2257f61a98f6, user_id=e4c0d774e24f49c680eb85fa5132e2d5, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.433489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.470773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.471079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.471450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.484219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 822/1658] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:26 2022] PUT /identity/v3/projects/5c3cf3e1edc8443cb8bb2257f61a98f6/users/e4c0d774e24f49c680eb85fa5132e2d5/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:26.514739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] MFA Rules not processed for user `e4c0d774e24f49c680eb85fa5132e2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:26.515984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.657975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 837/1659] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:26.671188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:26.673076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:26.830334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec72f7f0-7f89-4ffd-9727-99e82c85519f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:26.842723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fdafe0-3bd1-4cd6-84d5-0f0f9b0ed165 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:26.846918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdafe0-3bd1-4cd6-84d5-0f0f9b0ed165 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:26.847426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdafe0-3bd1-4cd6-84d5-0f0f9b0ed165 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:26.848861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fdafe0-3bd1-4cd6-84d5-0f0f9b0ed165 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:26.902162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdafe0-3bd1-4cd6-84d5-0f0f9b0ed165 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:26.904394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fdafe0-3bd1-4cd6-84d5-0f0f9b0ed165 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:26.922834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 823/1660] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:27.636732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:27.792979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28368300-3d0c-4a87-bd7e-5077c235ee97 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:27.799421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:27.800303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.800676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.801042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] PATH_INFO: `/v3/users/36fe4ddb2cf842e1b37bfa9f5210ff8b` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.802342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36fe4ddb2cf842e1b37bfa9f5210ff8b)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:27.804773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:27.868661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36fe4ddb2cf842e1b37bfa9f5210ff8b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.884620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36fe4ddb2cf842e1b37bfa9f5210ff8b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.886465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36fe4ddb2cf842e1b37bfa9f5210ff8b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.887632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36fe4ddb2cf842e1b37bfa9f5210ff8b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.896932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36fe4ddb2cf842e1b37bfa9f5210ff8b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:27.902196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:27.925055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 838/1661] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:27 2022] DELETE /identity/v3/users/36fe4ddb2cf842e1b37bfa9f5210ff8b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:27.929231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab2d7c-3808-4408-bc4a-c7ce7e92fadc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:27.930686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab2d7c-3808-4408-bc4a-c7ce7e92fadc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:27.931154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab2d7c-3808-4408-bc4a-c7ce7e92fadc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:27.971813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-88ab2d7c-3808-4408-bc4a-c7ce7e92fadc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:27.974637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-88ab2d7c-3808-4408-bc4a-c7ce7e92fadc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.148556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 824/1662] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:28.184880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:28.187879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:28.360940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712c866a-6c75-4cc9-bc3c-39bc2c858f54 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:28.367994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:28.368781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:28.369123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:28.369466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:28.392163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:28.394069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:28.411729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 839/1663] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:29.169929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.171135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.171571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.216591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] MFA Rules not processed for user `91a5130420764ac78383961d7665a964`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:29.217825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:29.354580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 825/1664] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:29.399793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.401363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:29.428017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97e5273-9979-4b6f-90cf-3fe8ba9a461a service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.442529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.443711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.444896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.445183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] PATH_INFO: `/v3/projects/407a94a20e5944078ba583a23c93a80d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.449213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=407a94a20e5944078ba583a23c93a80d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.454094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: WARNING py.warnings [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:29.454094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: warnings.warn(msg) Sep 13 19:50:29.454094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: Sep 13 19:50:29.455343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:29.630135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '407a94a20e5944078ba583a23c93a80d', 'request_id': 'req-f5cba1e9-595b-4123-b14a-b2a00d575e1f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:29.647331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '407a94a20e5944078ba583a23c93a80d', 'request_id': 'req-f5cba1e9-595b-4123-b14a-b2a00d575e1f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:29.669574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '407a94a20e5944078ba583a23c93a80d', 'request_id': 'req-f5cba1e9-595b-4123-b14a-b2a00d575e1f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:29.729539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] The token cache is being invalidate because project 407a94a20e5944078ba583a23c93a80d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:29.732784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:29.735450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f5cba1e9-595b-4123-b14a-b2a00d575e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:29.741008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 840/1665] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:29 2022] DELETE /identity/v3/projects/407a94a20e5944078ba583a23c93a80d => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:29.746149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:29.764779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe036b4-1bab-4c12-8a3e-ac693c7bf8e7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:29.790149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:29.926158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:29.927185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:29.927682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:29.928146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] PATH_INFO: `/v3/users/e4c0d774e24f49c680eb85fa5132e2d5` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:29.936242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4c0d774e24f49c680eb85fa5132e2d5)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:29.938948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: WARNING py.warnings [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 19:50:29.938948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: warnings.warn(msg) Sep 13 19:50:29.938948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: Sep 13 19:50:29.945764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:30.079020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4c0d774e24f49c680eb85fa5132e2d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:30.095358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4c0d774e24f49c680eb85fa5132e2d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:30.096591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4c0d774e24f49c680eb85fa5132e2d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:30.099240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4c0d774e24f49c680eb85fa5132e2d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:30.128251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-adaa381c-e7a7-4539-8827-83a5b1a61ef8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4c0d774e24f49c680eb85fa5132e2d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:30.186798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 826/1666] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:29 2022] DELETE /identity/v3/users/e4c0d774e24f49c680eb85fa5132e2d5 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:30.496096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb84fe-72ca-41df-a57d-ee0cd5c5fcfd None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.497201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb84fe-72ca-41df-a57d-ee0cd5c5fcfd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.497272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71eb84fe-72ca-41df-a57d-ee0cd5c5fcfd None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.499815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 841/1667] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 19:50:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:30.528741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d753d1-89ea-4a4d-b387-8dbf967d644b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.529051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d753d1-89ea-4a4d-b387-8dbf967d644b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.529296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d753d1-89ea-4a4d-b387-8dbf967d644b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.616166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-75d753d1-89ea-4a4d-b387-8dbf967d644b None None] MFA Rules not processed for user `36fab0873915497cada62e8ea73bd4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:30.619946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-75d753d1-89ea-4a4d-b387-8dbf967d644b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:30.724118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 827/1668] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:30.753983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb84fe-72ca-41df-a57d-ee0cd5c5fcfd None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:30.755952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-71eb84fe-72ca-41df-a57d-ee0cd5c5fcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:30.780899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71eb84fe-72ca-41df-a57d-ee0cd5c5fcfd None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:30.791815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:30.792599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:30.792985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:30.793914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:30.811542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:30.813212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:30.822784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 842/1669] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:31.214375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717818b-e97a-45a1-89e9-cf6e310aa6c1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.215366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717818b-e97a-45a1-89e9-cf6e310aa6c1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.215448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717818b-e97a-45a1-89e9-cf6e310aa6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.219362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.220677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.303200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9717818b-e97a-45a1-89e9-cf6e310aa6c1 None None] MFA Rules not processed for user `0792c6c88f9c4534aad5bf7ea3a4ac60`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:31.304597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9717818b-e97a-45a1-89e9-cf6e310aa6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.352424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:31.420552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee435f-e751-4e49-adc0-91ab0b310097 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:31.425818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:31.427039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.427644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.428120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] PATH_INFO: `/v3/projects/5c3cf3e1edc8443cb8bb2257f61a98f6` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.428974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:31.430607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c3cf3e1edc8443cb8bb2257f61a98f6)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:31.433298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:31.528858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 828/1670] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:31.543527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.543797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.543979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.562636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c3cf3e1edc8443cb8bb2257f61a98f6', 'request_id': 'req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:31.577514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c3cf3e1edc8443cb8bb2257f61a98f6', 'request_id': 'req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:31.582748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] MFA Rules not processed for user `0792c6c88f9c4534aad5bf7ea3a4ac60`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:31.583969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.609807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c3cf3e1edc8443cb8bb2257f61a98f6', 'request_id': 'req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:31.643066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] The token cache is being invalidate because project 5c3cf3e1edc8443cb8bb2257f61a98f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:31.645017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:31.653601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de94ca2e-a37c-470e-ae8c-fb333b27ee7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:31.653601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 843/1671] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:31 2022] DELETE /identity/v3/projects/5c3cf3e1edc8443cb8bb2257f61a98f6 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:31.654859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:31.654859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:31.654859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:31.726282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:31.727487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.776095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 829/1672] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:31.790368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.794993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:31.935332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 844/1673] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:31.936299 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:31.937591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:32.030022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f29c50-ab83-444d-b3f6-c5a7a30f323d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.049297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.050084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.050306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.050574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.056930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:32.101439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.103609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2949e4-b46b-4ecf-96cc-ead2ac5c2c5d service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.132319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 830/1674] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:32.133897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.134184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.134455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.144023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660907b-3ff5-4fd8-b744-ee4bf1cc4894 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.162335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.162970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.163251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.163521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.165307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:32.188343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:32.189747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:32.210276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.212006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.235550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 831/1675] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:32.239131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.264132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd241a-b7f6-46ff-828e-1b7be759af77 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.269050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 845/1676] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:32.291833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.292468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.292744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.296602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.297356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:32.297865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.302047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.350467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 832/1677] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:32.361824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.375643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e70bb-6ee7-477d-8913-1df801a6d9d8 service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.388650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.389285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.389567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.389906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.390549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.392816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.430922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:32.465785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 846/1678] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:32 2022] POST /identity/v3/projects => generated 355 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:32.466767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.491192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfb843e-8499-48a5-834c-25aca1a4185a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.558304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:32.637902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.638736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.639072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.639403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.640114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.648665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.730411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 833/1679] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:32 2022] POST /identity/v3/users => generated 278 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:32.733858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.750606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593f872-753c-405d-82d5-d321292b6f67 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.771280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.771911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.772195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.772470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.773668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.774781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.804405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 847/1680] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:32.809134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.830669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a41125-28bc-42d4-89dc-b576cfa818d6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.864369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce6468-f664-4fde-b4cf-56cc74b2c7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.865153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6468-f664-4fde-b4cf-56cc74b2c7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.865528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6468-f664-4fde-b4cf-56cc74b2c7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.866283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce6468-f664-4fde-b4cf-56cc74b2c7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.866867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce6468-f664-4fde-b4cf-56cc74b2c7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.868977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce6468-f664-4fde-b4cf-56cc74b2c7c2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:32.885538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 834/1681] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:32 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:32.889775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:32.905187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4949485-6c19-420d-b132-c5e9099b6124 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:32.918480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:32.920709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:32.921399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:32.922484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] PATH_INFO: `/v3/projects/89f4c563630d406ebae7aedc3d2a9ac7/users/58f5a69032ab45babbc599aac755d2f9/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:32.984076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89f4c563630d406ebae7aedc3d2a9ac7, user_id=58f5a69032ab45babbc599aac755d2f9, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:32.987307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:33.031059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 848/1682] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:32 2022] PUT /identity/v3/projects/89f4c563630d406ebae7aedc3d2a9ac7/users/58f5a69032ab45babbc599aac755d2f9/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:33.045894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.048870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.049770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.151524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] MFA Rules not processed for user `58f5a69032ab45babbc599aac755d2f9`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:33.156454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:33.397450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 835/1683] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:33.400973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:33.568469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d767344c-cb16-4f36-aa03-f0a1d8e79722 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:33.586586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:33.587636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.588065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.588475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.589271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:33.601413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:33.672688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 849/1684] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/projects => generated 351 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:33.676422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:33.702411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ed946-caf0-4685-a5c7-33856335e4bf None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:33.872659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:33.876422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.876830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:33.879025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:33.879729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:33.889931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:33.936235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 836/1685] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:33 2022] POST /identity/v3/users => generated 282 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:33.940953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:33.975088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda959f3-ce5a-40ae-96b8-6ef5589d23f6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:33.994019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:33.996580 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:33.998557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.005927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.007763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.009939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.062245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 850/1686] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:34.066956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.085406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fe326-a8d4-4c75-8783-fb9ceed26a72 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.099075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.100232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.100522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.100800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] PATH_INFO: `/v3/projects/163dc1f58e304399b4cf295b7c0dcc45/users/1b965ec2665e44ae99af89f423b303d7/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.152746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163dc1f58e304399b4cf295b7c0dcc45, user_id=1b965ec2665e44ae99af89f423b303d7, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.155467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.180571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 837/1687] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:34 2022] PUT /identity/v3/projects/163dc1f58e304399b4cf295b7c0dcc45/users/1b965ec2665e44ae99af89f423b303d7/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:34.181520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.198139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e15129-c081-46e6-b466-4aadadfb86ef admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.347315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a6e26-0afe-45d1-bebe-af00374ec021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.347944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a6e26-0afe-45d1-bebe-af00374ec021 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.348221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a6e26-0afe-45d1-bebe-af00374ec021 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.348537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a6e26-0afe-45d1-bebe-af00374ec021 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.349182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a6e26-0afe-45d1-bebe-af00374ec021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.355578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a6e26-0afe-45d1-bebe-af00374ec021 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.387626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 851/1688] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:34.389178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.391870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.392368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.392836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.400872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:34.433389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7827bdd9-0505-4ca3-a30c-b2ac2afd8205 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:34.448738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcca957-0224-453b-9223-076131b319b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:34.449978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca957-0224-453b-9223-076131b319b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.450266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca957-0224-453b-9223-076131b319b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.450536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcca957-0224-453b-9223-076131b319b4 admin admin] PATH_INFO: `/v3/projects/163dc1f58e304399b4cf295b7c0dcc45/users/1b965ec2665e44ae99af89f423b303d7/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.456173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcca957-0224-453b-9223-076131b319b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=163dc1f58e304399b4cf295b7c0dcc45, user_id=1b965ec2665e44ae99af89f423b303d7, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:34.462762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] MFA Rules not processed for user `eff856b09e5944a9bdcf7374cd9d4a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:34.463903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcca957-0224-453b-9223-076131b319b4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:34.464499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.494772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 838/1689] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:34 2022] PUT /identity/v3/projects/163dc1f58e304399b4cf295b7c0dcc45/users/1b965ec2665e44ae99af89f423b303d7/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:34.505281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:34.505563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:34.505978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:34.552682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:34.581041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] MFA Rules not processed for user `1b965ec2665e44ae99af89f423b303d7`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:34.582428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.694236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 852/1690] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:34.715884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.721146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 839/1691] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:34 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:34.806652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:34.809148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:34.900214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2826d932-cfab-490a-b29d-4230de0a34f8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:35.014255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffecc46-5f7f-4659-998d-7022ebfcf6c9 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:35.023385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:35.024426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:35.024912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:35.025422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.061768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:35.064457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:35.074916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d2256-c22c-4f89-a025-8a9ab3760dc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:35.076728 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2256-c22c-4f89-a025-8a9ab3760dc4 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:35.078144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2256-c22c-4f89-a025-8a9ab3760dc4 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:35.078530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d2256-c22c-4f89-a025-8a9ab3760dc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.114691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 840/1692] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:34 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:35.242868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:35.248339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d2256-c22c-4f89-a025-8a9ab3760dc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:35.257215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d2256-c22c-4f89-a025-8a9ab3760dc4 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:35.284266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 853/1693] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 569 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:35.305643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:35.307451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:35.335165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa20af1a-6bc1-4ad1-af82-602d588d1d3c service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:35.347333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c3c888-1770-4dc8-b9d2-31015da32bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:35.348081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3c888-1770-4dc8-b9d2-31015da32bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:35.348374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3c888-1770-4dc8-b9d2-31015da32bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:35.348746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c3c888-1770-4dc8-b9d2-31015da32bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.489693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:35.490320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:35.490784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.499031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3c888-1770-4dc8-b9d2-31015da32bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:35.504951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c3c888-1770-4dc8-b9d2-31015da32bb8 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:35.562818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 841/1694] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:35.593125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] MFA Rules not processed for user `53fa55dda30e4aaa86ae599e4df2fed3`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:35.594535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:35.769770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 854/1695] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:35.779807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195fa96-c2a3-4ae6-8721-cb3a17efbed9 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:35.780098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195fa96-c2a3-4ae6-8721-cb3a17efbed9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:35.780380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8195fa96-c2a3-4ae6-8721-cb3a17efbed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:35.838641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8195fa96-c2a3-4ae6-8721-cb3a17efbed9 None None] MFA Rules not processed for user `53fa55dda30e4aaa86ae599e4df2fed3`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:35.856847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8195fa96-c2a3-4ae6-8721-cb3a17efbed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:35.901977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 842/1696] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:35 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:36.230795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:36.232772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:36.266404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95f7861-c7dc-400f-86d5-c270f8cf925b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:36.274461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:36.275708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:36.276189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:36.276669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:36.302255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:36.304375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:36.342356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 855/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:36 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:36.519135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376347d8-7918-47ad-80f7-b6c53ea864d2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:36.520020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376347d8-7918-47ad-80f7-b6c53ea864d2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:36.520512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376347d8-7918-47ad-80f7-b6c53ea864d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:36.571726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:36.661640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-376347d8-7918-47ad-80f7-b6c53ea864d2 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:36.669708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-376347d8-7918-47ad-80f7-b6c53ea864d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:36.866881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 843/1698] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:36.986195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:36.988591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:37.171376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608997b8-2be4-4447-a8d2-85bcea76fc89 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.171717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608997b8-2be4-4447-a8d2-85bcea76fc89 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.172049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608997b8-2be4-4447-a8d2-85bcea76fc89 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.181954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 844/1699] 10.222.0.15 () {54 vars in 912 bytes} [Tue Sep 13 19:50:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:37.186907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3780a535-fa17-43e1-9fb5-804d0b6e27d3 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:37.188198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.188765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.189191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.202171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:37.203093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.203531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.203968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:37.225995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:37.227992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:37.264426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 856/1700] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 19:50:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:50:37.307479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] MFA Rules not processed for user `36fab0873915497cada62e8ea73bd4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:37.321205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:37.818606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 845/1701] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:50:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 632 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:37.870129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:37.873935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:37.910593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b82e55-7262-4d50-bd21-4393347e8ceb service swift] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:37.934533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8b946-49f7-4b37-b2cb-d7997647e350 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:37.936139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8b946-49f7-4b37-b2cb-d7997647e350 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:37.937645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8b946-49f7-4b37-b2cb-d7997647e350 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:37.938521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8b946-49f7-4b37-b2cb-d7997647e350 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:38.009204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8b946-49f7-4b37-b2cb-d7997647e350 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:38.013444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8b946-49f7-4b37-b2cb-d7997647e350 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:38.053808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 857/1702] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:38.329983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:38.331413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:38.567739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6fbc74-712e-436c-b9e3-6493990576d2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:38.603079 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:38.604521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:38.605687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:38.612904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] PATH_INFO: `/v3/users/4c46716661414f74b3ed33c302d116df` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:38.624393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c46716661414f74b3ed33c302d116df)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:38.627383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:38.727756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:38.785166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c46716661414f74b3ed33c302d116df'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:38.829504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c46716661414f74b3ed33c302d116df'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:38.832397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c46716661414f74b3ed33c302d116df'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:38.832975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c46716661414f74b3ed33c302d116df'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:38.851937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c46716661414f74b3ed33c302d116df'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:38.932332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 846/1703] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:38 2022] DELETE /identity/v3/users/4c46716661414f74b3ed33c302d116df => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:38.933966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:38.934481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:38.934575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:38.993240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:38.995070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:39.056977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:39.172636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 858/1704] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:39.193776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:39.198049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:39.353100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9f1e7-9cf5-4442-bada-4b70d938bbf9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:39.363235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae2342-447a-4078-ad37-521d72e4d9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:39.363880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae2342-447a-4078-ad37-521d72e4d9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:39.364167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae2342-447a-4078-ad37-521d72e4d9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:39.364442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae2342-447a-4078-ad37-521d72e4d9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:39.395564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae2342-447a-4078-ad37-521d72e4d9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:39.397854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ae2342-447a-4078-ad37-521d72e4d9e5 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:39.428368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 847/1705] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:40.515852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:40.517396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:40.549162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b832ab4-9f40-4a64-91e2-2401866a73dc None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:40.566703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:40.567793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:40.568351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:40.569660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] PATH_INFO: `/v3/users/6744d5ca67184164bacc6eb19dae000f` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:40.577409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6744d5ca67184164bacc6eb19dae000f)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:40.580300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:40.682618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6744d5ca67184164bacc6eb19dae000f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.700624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6744d5ca67184164bacc6eb19dae000f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.701725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6744d5ca67184164bacc6eb19dae000f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.702668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6744d5ca67184164bacc6eb19dae000f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.719967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-48e83026-1ba7-41d1-8cf1-b1d1ca6d30f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6744d5ca67184164bacc6eb19dae000f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:40.760563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 859/1706] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:40 2022] DELETE /identity/v3/users/6744d5ca67184164bacc6eb19dae000f => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:40.765920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:40.766405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:40.766685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:40.818743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:40.819861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.059794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 848/1707] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:41.061933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d0d83-0667-42a5-94a4-5514856633b4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.062344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d0d83-0667-42a5-94a4-5514856633b4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.062734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621d0d83-0667-42a5-94a4-5514856633b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.116656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-621d0d83-0667-42a5-94a4-5514856633b4 None None] MFA Rules not processed for user `53fa55dda30e4aaa86ae599e4df2fed3`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:41.119744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-621d0d83-0667-42a5-94a4-5514856633b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.195042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:41.197767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.388743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 860/1708] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:41.392587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.393113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.395316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.453724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] MFA Rules not processed for user `36fab0873915497cada62e8ea73bd4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:41.472898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.491200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8449a-e988-42ec-b257-154b6f5d4dd7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:41.499663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 861/1709] 10.222.0.61 () {58 vars in 1015 bytes} [Tue Sep 13 19:50:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:41.507672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:41.508527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.508936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.509318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.522235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:41.525433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:41.551440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:41.562221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:41.564206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:41.569120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a8771-5b2f-4c18-a089-165b827f4dc3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:41.579997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:41.581292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:41.583056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:41.583492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:41.600584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:41.602417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:41.622279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 849/1710] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:41.626251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 862/1711] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:42.899308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:42.900994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:42.943593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf80ff4-91d2-4ee1-8fd3-d885533970de service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:42.960924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:42.961625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:42.964271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:42.964595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] PATH_INFO: `/v3/projects/77649d227cf34946a5a47a86c1c3e7ef` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:42.968522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77649d227cf34946a5a47a86c1c3e7ef)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:42.978951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:43.161625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77649d227cf34946a5a47a86c1c3e7ef', 'request_id': 'req-3228b5e9-5c9f-4989-8375-6d37f3123331'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.191604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77649d227cf34946a5a47a86c1c3e7ef', 'request_id': 'req-3228b5e9-5c9f-4989-8375-6d37f3123331'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.209752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77649d227cf34946a5a47a86c1c3e7ef', 'request_id': 'req-3228b5e9-5c9f-4989-8375-6d37f3123331'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.257369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] The token cache is being invalidate because project 77649d227cf34946a5a47a86c1c3e7ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:43.259848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.262512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3228b5e9-5c9f-4989-8375-6d37f3123331 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.266706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 850/1712] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:42 2022] DELETE /identity/v3/projects/77649d227cf34946a5a47a86c1c3e7ef => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:43.270090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:43.287996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d82eb-24ff-4bcf-aa76-f6a0c6c0bf68 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:43.332785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:43.477185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:43.479508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:43.480943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:43.482048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] PATH_INFO: `/v3/users/3ff93ca22f8e478596d3bca6a8cf3abd` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:43.487366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ff93ca22f8e478596d3bca6a8cf3abd)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:43.496694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:43.622222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ff93ca22f8e478596d3bca6a8cf3abd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.638822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ff93ca22f8e478596d3bca6a8cf3abd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.646362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ff93ca22f8e478596d3bca6a8cf3abd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.647768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ff93ca22f8e478596d3bca6a8cf3abd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.667481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f6961b4f-b536-474f-876c-0d61f5b198a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ff93ca22f8e478596d3bca6a8cf3abd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:43.743625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 863/1713] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:43 2022] DELETE /identity/v3/users/3ff93ca22f8e478596d3bca6a8cf3abd => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:44.114477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e8523-e12b-45cc-853e-225ec7b9beb9 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.115062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e8523-e12b-45cc-853e-225ec7b9beb9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.115157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7e8523-e12b-45cc-853e-225ec7b9beb9 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.118942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 851/1714] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:50:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:44.136102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7256f-c918-45e9-a14c-f619004d0af3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.136394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7256f-c918-45e9-a14c-f619004d0af3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.136836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb7256f-c918-45e9-a14c-f619004d0af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.204546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9cb7256f-c918-45e9-a14c-f619004d0af3 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:44.208229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9cb7256f-c918-45e9-a14c-f619004d0af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:44.409236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 864/1715] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:44.565047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e8523-e12b-45cc-853e-225ec7b9beb9 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:44.568707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cc7e8523-e12b-45cc-853e-225ec7b9beb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:44.733503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.733904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.734257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.798249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] MFA Rules not processed for user `eff856b09e5944a9bdcf7374cd9d4a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:44.803536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:44.879085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7e8523-e12b-45cc-853e-225ec7b9beb9 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:44.899327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:44.899867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:44.900125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:44.906080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] PATH_INFO: `/v3/users/894ca5b657fe4483935459ef7989244f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:44.907745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=894ca5b657fe4483935459ef7989244f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:44.910349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:45.014884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894ca5b657fe4483935459ef7989244f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:45.040634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894ca5b657fe4483935459ef7989244f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:45.043971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894ca5b657fe4483935459ef7989244f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:45.044893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894ca5b657fe4483935459ef7989244f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:45.078280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d6f48a21-7142-4ef1-b6cd-7191db4ddbf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894ca5b657fe4483935459ef7989244f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:45.112577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:45.147392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:45.173286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 852/1716] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:44 2022] DELETE /identity/v3/users/894ca5b657fe4483935459ef7989244f => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:45.202132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:45.208055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:45.208678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:45.243760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 865/1717] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 512 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:45.332275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:45.346782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:45.549674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 853/1718] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:45.661007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.661007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:45.667965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:45.670568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:45.911021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38982f04-cc3c-4d33-b119-8e619d7c664c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:46.017524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bccaff-4759-48f0-b859-f24e7a8c8ea7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:46.042147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:46.044914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:46.047034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:46.047580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:46.056882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:46.091793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c081212-b70a-489b-a5b7-c3537938ace7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:46.094602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c081212-b70a-489b-a5b7-c3537938ace7 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:46.095923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c081212-b70a-489b-a5b7-c3537938ace7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:46.096215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c081212-b70a-489b-a5b7-c3537938ace7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:46.112821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:46.121213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:46.137117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 854/1719] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:46.152543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c081212-b70a-489b-a5b7-c3537938ace7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:46.152543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c081212-b70a-489b-a5b7-c3537938ace7 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:46.165529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 866/1720] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:47.029976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:47.033834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.055243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccfe1aa-122f-4c64-a153-1c921c9a9730 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:47.061490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:47.068265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.069942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.070751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] PATH_INFO: `/v3/projects/6991de03924d4af2a160e795cad82e5b` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.074812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6991de03924d4af2a160e795cad82e5b)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:47.077911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:47.249899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6991de03924d4af2a160e795cad82e5b', 'request_id': 'req-4340d043-7fa0-4919-b61a-5d57c83ca789'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.280984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6991de03924d4af2a160e795cad82e5b', 'request_id': 'req-4340d043-7fa0-4919-b61a-5d57c83ca789'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.314551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6991de03924d4af2a160e795cad82e5b', 'request_id': 'req-4340d043-7fa0-4919-b61a-5d57c83ca789'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.333693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] The token cache is being invalidate because project 6991de03924d4af2a160e795cad82e5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:47.336984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.338344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4340d043-7fa0-4919-b61a-5d57c83ca789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.340485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 855/1721] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:47 2022] DELETE /identity/v3/projects/6991de03924d4af2a160e795cad82e5b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:47.347742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f322cfd-af42-4b88-a58b-d7afbc100115 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.348897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f322cfd-af42-4b88-a58b-d7afbc100115 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.349423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f322cfd-af42-4b88-a58b-d7afbc100115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.359371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab6cd6-8518-4316-a92b-3a133e342896 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.359636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab6cd6-8518-4316-a92b-3a133e342896 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.362748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aab6cd6-8518-4316-a92b-3a133e342896 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.374756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 856/1722] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:50:47 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:50:47.385972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab6cd6-8518-4316-a92b-3a133e342896 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:47.388599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2aab6cd6-8518-4316-a92b-3a133e342896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.408066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7f322cfd-af42-4b88-a58b-d7afbc100115 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:47.409929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:47.419617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7f322cfd-af42-4b88-a58b-d7afbc100115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.696830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aab6cd6-8518-4316-a92b-3a133e342896 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:47.709114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:47.711944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.712466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.713285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] PATH_INFO: `/v3/projects/12bfdb3ea95a4cefa8f3c01be875ef14` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.723846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12bfdb3ea95a4cefa8f3c01be875ef14)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:47.725888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 867/1723] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:47.727563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b8244-cfb5-42ad-916e-9f99d9f9b3f0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.728527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:47.729031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b8244-cfb5-42ad-916e-9f99d9f9b3f0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.731589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8b8244-cfb5-42ad-916e-9f99d9f9b3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.779118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8e8b8244-cfb5-42ad-916e-9f99d9f9b3f0 None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:47.780541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8e8b8244-cfb5-42ad-916e-9f99d9f9b3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.848459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12bfdb3ea95a4cefa8f3c01be875ef14', 'request_id': 'req-77cad615-4268-444a-805f-def511b2b3e9'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.859658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12bfdb3ea95a4cefa8f3c01be875ef14', 'request_id': 'req-77cad615-4268-444a-805f-def511b2b3e9'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.878139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:47.879289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12bfdb3ea95a4cefa8f3c01be875ef14', 'request_id': 'req-77cad615-4268-444a-805f-def511b2b3e9'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.900973 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] The token cache is being invalidate because project 12bfdb3ea95a4cefa8f3c01be875ef14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:47.902191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.905742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-77cad615-4268-444a-805f-def511b2b3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:47.911646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 857/1724] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:47 2022] DELETE /identity/v3/projects/12bfdb3ea95a4cefa8f3c01be875ef14 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:47.913164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.913593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.913994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:47.982227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 868/1725] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:47.986204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:47.987405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:47.987818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:47.988650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:47.989055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.072543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.075767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.146813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 858/1726] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.150004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.156259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.189944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0884a7-6f25-469a-b078-c011fddde1f1 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.209377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.223240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.223760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.224324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] PATH_INFO: `/v3/projects/31a694d77d3c4c21bccde9920a14dc88` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.227034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a694d77d3c4c21bccde9920a14dc88)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.230978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.320757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:48.325272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 869/1727] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.327196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.371940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a694d77d3c4c21bccde9920a14dc88', 'request_id': 'req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.375740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee0314-a78f-4b06-bca0-c057fae0fbed None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.398205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a694d77d3c4c21bccde9920a14dc88', 'request_id': 'req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.418297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a694d77d3c4c21bccde9920a14dc88', 'request_id': 'req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.463301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] The token cache is being invalidate because project 31a694d77d3c4c21bccde9920a14dc88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:48.464394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.467342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9da36c14-d28a-42fe-9bf5-fcc3bc2a9266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:48.489656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 859/1728] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:48 2022] DELETE /identity/v3/projects/31a694d77d3c4c21bccde9920a14dc88 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:48.506575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.507328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.507710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.585038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.591456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.614626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.615279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.618785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.619381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.622608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:48.624010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.631492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db702da-8946-4a05-9142-9f8dbfb95a96 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.667992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 870/1729] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:48.674235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.674888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.675417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.744242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.746105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.785091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 871/1730] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.786337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.789990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:48.815791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529b89c-101a-4582-9c46-5b56ebefa160 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.821115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a006f-9364-4368-822d-dca0ceb47c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:48.821958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a006f-9364-4368-822d-dca0ceb47c44 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.822391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a006f-9364-4368-822d-dca0ceb47c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.822814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a006f-9364-4368-822d-dca0ceb47c44 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.823650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a006f-9364-4368-822d-dca0ceb47c44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:48.826126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a006f-9364-4368-822d-dca0ceb47c44 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:48.899398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 872/1731] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:48.899398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:48.899723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:48.900001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:48.902598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 860/1732] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:48.907554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:48.936242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1bf1c-d70f-4616-a7cd-d5d83ca154b3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:48.962020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:48.963284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.060474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:49.124627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 873/1733] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:49.125869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.128163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.128884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.130794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.131085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.133224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.138216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e54537-e4c4-4f9a-b7f7-4f4bf4325b26 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.138838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.146279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.147114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.147542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.147951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.148737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:49.149185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.163527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.183040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 874/1734] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:49.184358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.195380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 861/1735] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:48 2022] POST /identity/v3/users => generated 284 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:49.195380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.208940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794f5e5-dc36-4a6b-b2f0-5d33d0303fad admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.216012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.217962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.218488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.218970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.220072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:49.220603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.222679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.228952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95b273-9a08-4a82-8508-1ff5559ca950 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.248316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 875/1736] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:49.251497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.267478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.268426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.268920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.269381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.270308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.270902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dda77-d102-4151-a2f5-d2e580adf7a9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.278401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.278749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.279246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.279664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.280049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.283388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.283388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.303051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 876/1737] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:49.304597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.328339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49771a9-9f83-43ca-b679-4b9762e37f04 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.335355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.336470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.336810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.337262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.338118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.341365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.368545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 862/1738] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/projects => generated 381 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:49.375110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.396446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 877/1739] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:49.408156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.451498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:49.524642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7499fd-1bf6-4099-b5ad-050a05c6fb75 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.564210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65e7f78-0e0a-4541-9641-92b30632ef02 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.570134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.570995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.571419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.571816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.572595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.576518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.610560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 878/1740] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:49.612607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.633002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def24c9a-69fd-4f88-b6d8-d9b31f73511b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.641266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.642701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.643176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.643587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.644681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.648042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.678720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.679464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.679794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.680174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.680907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.683324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.699272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 879/1741] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:49 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:49.700246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.712348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 863/1742] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:49.712348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.717079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62be61-231e-476b-8a19-03cea6b0a34a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.724208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.724841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.725345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.725752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.727018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.729787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.741999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 880/1743] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:49.742828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.754057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc4f6c-fb12-4025-a678-ef2b86833b99 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.755713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24c1e1-55ae-4a16-895c-514c1e6cec03 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.761491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.762232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.762510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.763410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] PATH_INFO: `/v3/projects/7ef0207478924654b32aa9a5eb4ff293/users/86834cc453eb4c688716388d12987580/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.766907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.767842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.767842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.768177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.769077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.770483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.793630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 864/1744] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:49.798525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.805849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ef0207478924654b32aa9a5eb4ff293, user_id=86834cc453eb4c688716388d12987580, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.813947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.819700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273edd92-02ef-4d96-b868-cfd3591c1d8b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:49.830817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:49.831920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:49.832195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:49.832472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:49.833201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:49.840300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:49.841826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 881/1745] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:49 2022] PUT /identity/v3/projects/7ef0207478924654b32aa9a5eb4ff293/users/86834cc453eb4c688716388d12987580/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:49.842801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.866874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 865/1746] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:49.866874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:49.867398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:49.886829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.019211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25336f5c-a4c2-4152-af60-d127ea208abf admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.024947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a93b847-cbf9-4e2f-9883-d83c006eef80 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.040848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd11f77b-4aed-4ff3-854e-0bcdeade3414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.042362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11f77b-4aed-4ff3-854e-0bcdeade3414 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.042799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11f77b-4aed-4ff3-854e-0bcdeade3414 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.046144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd11f77b-4aed-4ff3-854e-0bcdeade3414 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.048996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11f77b-4aed-4ff3-854e-0bcdeade3414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.051902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd11f77b-4aed-4ff3-854e-0bcdeade3414 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.070051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 882/1747] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:50.075220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.075530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.075776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.132876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] MFA Rules not processed for user `86834cc453eb4c688716388d12987580`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:50.136083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.139237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.212152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.215616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.216200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.217062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.225662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.258306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.261876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.279839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 866/1748] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:50.280867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.300712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b28a6e-8c9b-4d38-a585-489a941c4304 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.308594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.309533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.310140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.310393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] PATH_INFO: `/v3/projects/c6559ba3f88a42febc349d20b5e7abf6/users/e803e271f54943c98c3bc1f867d95ef2/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.348134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6559ba3f88a42febc349d20b5e7abf6, user_id=e803e271f54943c98c3bc1f867d95ef2, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.352906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.369523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 883/1749] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:50.373634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.385013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 867/1750] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:50 2022] PUT /identity/v3/projects/c6559ba3f88a42febc349d20b5e7abf6/users/e803e271f54943c98c3bc1f867d95ef2/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:50.387464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:50.412768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e802149-3cda-4528-beed-feb040c4c387 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.667235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953969c8-741e-4088-b550-47505c162cc4 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:50.673897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95e0d4c-4c91-4a14-8215-7073c28151d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.674518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e0d4c-4c91-4a14-8215-7073c28151d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.678699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e0d4c-4c91-4a14-8215-7073c28151d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.678992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95e0d4c-4c91-4a14-8215-7073c28151d0 admin admin] PATH_INFO: `/v3/projects/08a1ab177d4140788ef43f270491e970/users/0c0d76555c4743e0a37890429260ff61/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.694663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec50e0-6ef5-409e-9916-b29038de0255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:50.697463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec50e0-6ef5-409e-9916-b29038de0255 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.697887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec50e0-6ef5-409e-9916-b29038de0255 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.698782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec50e0-6ef5-409e-9916-b29038de0255 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.699479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec50e0-6ef5-409e-9916-b29038de0255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.701383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ec50e0-6ef5-409e-9916-b29038de0255 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.735664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e0d4c-4c91-4a14-8215-7073c28151d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a1ab177d4140788ef43f270491e970, user_id=0c0d76555c4743e0a37890429260ff61, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:50.739160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95e0d4c-4c91-4a14-8215-7073c28151d0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:50.768989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 884/1751] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:50 2022] PUT /identity/v3/projects/08a1ab177d4140788ef43f270491e970/users/0c0d76555c4743e0a37890429260ff61/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:50.768989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.768989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.768989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.769488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 868/1752] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/projects => generated 369 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:50.771404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:50.771404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:50.771608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:50.797012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] MFA Rules not processed for user `e803e271f54943c98c3bc1f867d95ef2`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:50.798784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.813336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] MFA Rules not processed for user `0c0d76555c4743e0a37890429260ff61`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:50.815086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:50.818205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:50.890954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 885/1753] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:50.896076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.018969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 869/1754] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:51.020408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.046448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036be7e7-2e92-4f47-b61c-75931b440779 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.055491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.057659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.057991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.058263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.058910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.061349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.079769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:51.152095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 886/1755] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:50 2022] POST /identity/v3/users => generated 291 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:51.154717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.177507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a326b7bb-b225-4984-b600-86beb975d708 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.181205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531336-846c-4f31-bc3e-1e2d0a524d4a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.185896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.187054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.187378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.187841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.188823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.192700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.195276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.195998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.196347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.196790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.197648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.202640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.239945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 870/1756] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:51 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:51.240843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.295847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 887/1757] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:51 2022] POST /identity/v3/projects => generated 371 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:51.298182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.382550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c68f26-a4ca-4442-b6d1-482c25d87a3a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.480257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061dbb4d-b117-4df1-a492-1895515c6494 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.501959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.503119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.504171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.504665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.505693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.508217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.548768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 888/1758] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:51 2022] POST /identity/v3/users => generated 296 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:51.553146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.565580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.566102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.566558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.566873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.567496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.571251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.591452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d151a01-4d27-47a2-8301-0a44fc9d53c9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.603616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.611968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 871/1759] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 369 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:51.613786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.616175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.617097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.617393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.619572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.624999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.642415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9486bc1-2c5b-4684-8ead-808dcb314ef3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.646885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:51.662213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.662992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.663735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.666575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.670298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.672136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.679994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 889/1760] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:51 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:51.680944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.705225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 872/1761] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:51.708572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.714163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79de803a-55ee-4e34-9789-2c6c7ea0fb95 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.719128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.719805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.720258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.720538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] PATH_INFO: `/v3/projects/c7b4109314cc48cea6a213ac1d6ea0f8/users/9c55a3f8565b491e818fcd6942f433c1/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.735864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a37c-6c07-4b18-8f57-49326bc86794 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.741797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.742709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.743059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.743568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.744368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.747502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.768774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 873/1762] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:51.770175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.782903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b4109314cc48cea6a213ac1d6ea0f8, user_id=9c55a3f8565b491e818fcd6942f433c1, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.785662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.793633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626161bf-c30e-4786-ad79-45466c35d68f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:51.810815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:51.812775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:51.816993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:51.817129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 890/1763] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:51 2022] PUT /identity/v3/projects/c7b4109314cc48cea6a213ac1d6ea0f8/users/9c55a3f8565b491e818fcd6942f433c1/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:51.817219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] PATH_INFO: `/v3/projects/33b93841fe8b4aa29329f6c0e9e80ed2/users/cd888ca6c9354cdda997fbbe23b9e026/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:51.817306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.858660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b93841fe8b4aa29329f6c0e9e80ed2, user_id=cd888ca6c9354cdda997fbbe23b9e026, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:51.888256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:51.922768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 874/1764] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:51 2022] PUT /identity/v3/projects/33b93841fe8b4aa29329f6c0e9e80ed2/users/cd888ca6c9354cdda997fbbe23b9e026/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:51.922768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:51.944783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:51.953273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c507e-a09d-4d7f-9e5f-f47c405ce8f3 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.130763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2affd7f6-4c64-4062-a5c2-b6becb86f208 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.147375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.148397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.148499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.148863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.149993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.152578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.171859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 875/1765] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:51 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:52.171859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.189884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d8ed24-8aa2-4b2b-8cd3-4d1f4cc0d063 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.195200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.195200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.195200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.195200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] PATH_INFO: `/v3/projects/1d9a9c8b141844cd8f7c8a08f99b7bf8/users/5c10429db20d46b989402c2696f2c29e/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.198151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.198795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.199072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.199337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.200004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.201586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.212340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9a9c8b141844cd8f7c8a08f99b7bf8, user_id=5c10429db20d46b989402c2696f2c29e, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.212609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 876/1766] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:52.216069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.217576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.230933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d18c762-4f9b-4b14-b393-ee70066a3ef9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.240885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 891/1767] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:51 2022] PUT /identity/v3/projects/1d9a9c8b141844cd8f7c8a08f99b7bf8/users/5c10429db20d46b989402c2696f2c29e/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:52.242326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.246701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.247648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.247927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.248196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] PATH_INFO: `/v3/projects/c7b4109314cc48cea6a213ac1d6ea0f8/users/9c55a3f8565b491e818fcd6942f433c1/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.252033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7b4109314cc48cea6a213ac1d6ea0f8, user_id=9c55a3f8565b491e818fcd6942f433c1, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.254717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.285266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 877/1768] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:52 2022] PUT /identity/v3/projects/c7b4109314cc48cea6a213ac1d6ea0f8/users/9c55a3f8565b491e818fcd6942f433c1/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:52.286293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.367237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b2437d-23e4-4c2b-9bcf-a5f833be5b14 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.375715 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.456890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc82bce-3181-475a-9310-16a44f5e5842 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.462917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c56c7-e185-4969-870a-0e7793872cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.464723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c56c7-e185-4969-870a-0e7793872cb2 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.465170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c56c7-e185-4969-870a-0e7793872cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.465598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c56c7-e185-4969-870a-0e7793872cb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.466449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c56c7-e185-4969-870a-0e7793872cb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.470150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c56c7-e185-4969-870a-0e7793872cb2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.489257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 878/1769] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:52.491031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d358c63-b21d-4a4f-9565-936ce7d86b77 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.491527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d358c63-b21d-4a4f-9565-936ce7d86b77 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.492026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d358c63-b21d-4a4f-9565-936ce7d86b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.520564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.521246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.521515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.532343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] PATH_INFO: `/v3/projects/33b93841fe8b4aa29329f6c0e9e80ed2/users/cd888ca6c9354cdda997fbbe23b9e026/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.534965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=33b93841fe8b4aa29329f6c0e9e80ed2, user_id=cd888ca6c9354cdda997fbbe23b9e026, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.537658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.564963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9d358c63-b21d-4a4f-9565-936ce7d86b77 None None] MFA Rules not processed for user `9c55a3f8565b491e818fcd6942f433c1`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:52.566269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9d358c63-b21d-4a4f-9565-936ce7d86b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.579938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 892/1770] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:52 2022] PUT /identity/v3/projects/33b93841fe8b4aa29329f6c0e9e80ed2/users/cd888ca6c9354cdda997fbbe23b9e026/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:52.580943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.689056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.694695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 879/1771] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:52.696168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40569245-a8bc-4d9d-82f2-5282372cbb6e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.696455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40569245-a8bc-4d9d-82f2-5282372cbb6e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.696725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40569245-a8bc-4d9d-82f2-5282372cbb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.729238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c4ce-6b5f-4bd4-8244-54799d2fd0ed admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.734537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.735031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.735423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.735789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] PATH_INFO: `/v3/projects/1d9a9c8b141844cd8f7c8a08f99b7bf8/users/5c10429db20d46b989402c2696f2c29e/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.737857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d9a9c8b141844cd8f7c8a08f99b7bf8, user_id=5c10429db20d46b989402c2696f2c29e, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.740949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.751827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-40569245-a8bc-4d9d-82f2-5282372cbb6e None None] MFA Rules not processed for user `cd888ca6c9354cdda997fbbe23b9e026`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:52.753033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-40569245-a8bc-4d9d-82f2-5282372cbb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.756471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 893/1772] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:52 2022] PUT /identity/v3/projects/1d9a9c8b141844cd8f7c8a08f99b7bf8/users/5c10429db20d46b989402c2696f2c29e/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:52.757595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.759493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.830034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b168f3-2171-46c1-a24f-24e30bfba5b8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.833991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.835032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.835032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.835206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.840975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 880/1773] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:52.840975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.840975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.840975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.875600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] MFA Rules not processed for user `5c10429db20d46b989402c2696f2c29e`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:52.876727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.899571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:52.918566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.920131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.932058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 894/1774] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:52.933562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:52.935019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:52.951484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766ad6d4-d49a-4fae-9b9e-693a77bbb4a3 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:52.956347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:52.956961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:52.957241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:52.957512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:52.977330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:52.981137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:52.993415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 881/1775] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:52 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:52.993559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 895/1776] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:52 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:53.002302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.012785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a398f8-1014-4196-bfc6-5873f1ed6633 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.015714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.017123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.018925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.019160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.019508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.025630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0379f0-dadd-4fe7-9f8f-6b03b8364435 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.029815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.029950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.030410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.030683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.030973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.031225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.044748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 882/1777] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:53.095243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.096668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.106961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 896/1778] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:53.212383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.222496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d9c17-81e4-4ce8-a14e-2ca44a93598b service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.227156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.227766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.228046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.228402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.297753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:53.300044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.302010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.313291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 883/1779] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:53.509954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:53.524895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7741aab7-e6e3-4842-9a32-f7c4f2e9508f service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:53.532431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c2d955-e9fb-438b-9f35-a8ce73fa707c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:53.533492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2d955-e9fb-438b-9f35-a8ce73fa707c service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:53.533835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2d955-e9fb-438b-9f35-a8ce73fa707c service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:53.534072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2d955-e9fb-438b-9f35-a8ce73fa707c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:53.601384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2d955-e9fb-438b-9f35-a8ce73fa707c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:53.602977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c2d955-e9fb-438b-9f35-a8ce73fa707c service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:53.612672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 897/1780] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:56.739342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:56.815661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c54ed-1e86-4530-8958-455fa3379a37 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:56.821163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:56.821819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.822098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.822372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] PATH_INFO: `/v3/users/e803e271f54943c98c3bc1f867d95ef2` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.824015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e803e271f54943c98c3bc1f867d95ef2)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:56.826032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:56.881645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e803e271f54943c98c3bc1f867d95ef2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:56.891822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e803e271f54943c98c3bc1f867d95ef2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:56.892640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e803e271f54943c98c3bc1f867d95ef2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:56.893536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e803e271f54943c98c3bc1f867d95ef2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:56.902690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e803e271f54943c98c3bc1f867d95ef2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:56.924746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 884/1781] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:56 2022] DELETE /identity/v3/users/e803e271f54943c98c3bc1f867d95ef2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:56.931068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:56.931987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:56.932267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:56.954253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:56.955474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.023596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 898/1782] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:50:57.033689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.035113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.094535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd4d664-977a-4daf-9abe-2d4b73f68132 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.098305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c615d-5a1b-4309-8d0a-8276499cd8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.099057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c615d-5a1b-4309-8d0a-8276499cd8ce service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.099335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c615d-5a1b-4309-8d0a-8276499cd8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.099926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c615d-5a1b-4309-8d0a-8276499cd8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.109265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c615d-5a1b-4309-8d0a-8276499cd8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.110808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c615d-5a1b-4309-8d0a-8276499cd8ce service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.120588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 885/1783] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:57.211008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.212565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.226666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6b993-e261-44aa-bf4e-e0d5bda7c540 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.231425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.232314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.232747 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.233164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] PATH_INFO: `/v3/users/86834cc453eb4c688716388d12987580` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.235321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86834cc453eb4c688716388d12987580)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.238085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.246155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.302525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86834cc453eb4c688716388d12987580'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.313862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86834cc453eb4c688716388d12987580'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.314692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86834cc453eb4c688716388d12987580'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.315321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86834cc453eb4c688716388d12987580'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.323854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86834cc453eb4c688716388d12987580'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.356286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 899/1784] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:57 2022] DELETE /identity/v3/users/86834cc453eb4c688716388d12987580 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:57.360758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09761bf-6168-4130-b462-d8278358e113 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.361042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09761bf-6168-4130-b462-d8278358e113 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.361310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09761bf-6168-4130-b462-d8278358e113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.380497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f09761bf-6168-4130-b462-d8278358e113 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:57.381695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f09761bf-6168-4130-b462-d8278358e113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.394149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.448187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 886/1785] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:57.457903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.459345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.521240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09761bf-6168-4130-b462-d8278358e113 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.523364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f09761bf-6168-4130-b462-d8278358e113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.537134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09761bf-6168-4130-b462-d8278358e113 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.542910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.543739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.544035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284335b-5629-4e4f-9c00-4f780c992b99 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.544357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.544820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] PATH_INFO: `/v3/projects/c6559ba3f88a42febc349d20b5e7abf6` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.546264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6559ba3f88a42febc349d20b5e7abf6)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.548334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.549050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.549816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.550326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.550692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.562749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.564840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.578662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 900/1786] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:50:57.607315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6559ba3f88a42febc349d20b5e7abf6', 'request_id': 'req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.614964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6559ba3f88a42febc349d20b5e7abf6', 'request_id': 'req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.629942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6559ba3f88a42febc349d20b5e7abf6', 'request_id': 'req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.648100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] The token cache is being invalidate because project c6559ba3f88a42febc349d20b5e7abf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:57.648392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.649093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.651020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 887/1787] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:57 2022] DELETE /identity/v3/projects/c6559ba3f88a42febc349d20b5e7abf6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:57.654718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.666983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f68c4d-c92d-4f21-a343-94fd163cb8b4 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:57.746980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:57.747625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:57.748065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:57.748341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] PATH_INFO: `/v3/users/cd888ca6c9354cdda997fbbe23b9e026` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:57.749476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd888ca6c9354cdda997fbbe23b9e026)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:57.751488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:57.785272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:57.813767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd888ca6c9354cdda997fbbe23b9e026'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.822839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd888ca6c9354cdda997fbbe23b9e026'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.823538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd888ca6c9354cdda997fbbe23b9e026'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.824251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd888ca6c9354cdda997fbbe23b9e026'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.838951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd888ca6c9354cdda997fbbe23b9e026'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:57.861568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 901/1788] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:57 2022] DELETE /identity/v3/users/cd888ca6c9354cdda997fbbe23b9e026 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:57.944033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:57.946026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:57.972433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a6b66b-a90c-4e08-9fac-5d71a6e03e2d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.049681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.050461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.050776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.051049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] PATH_INFO: `/v3/projects/7ef0207478924654b32aa9a5eb4ff293` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.052215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ef0207478924654b32aa9a5eb4ff293)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.054211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.098780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.101397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.105791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.111769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ef0207478924654b32aa9a5eb4ff293', 'request_id': 'req-4fa81f90-4027-42cb-8200-47c385e74c3d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.122524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c062d-5693-4d53-87c5-636bba070de5 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.123777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ef0207478924654b32aa9a5eb4ff293', 'request_id': 'req-4fa81f90-4027-42cb-8200-47c385e74c3d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.140529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ef0207478924654b32aa9a5eb4ff293', 'request_id': 'req-4fa81f90-4027-42cb-8200-47c385e74c3d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.157113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] The token cache is being invalidate because project 7ef0207478924654b32aa9a5eb4ff293 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:58.157599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.158896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.160728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 888/1789] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:57 2022] DELETE /identity/v3/projects/7ef0207478924654b32aa9a5eb4ff293 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.166652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.169053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.203445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.208254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.208546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.208824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] PATH_INFO: `/v3/users/58f5a69032ab45babbc599aac755d2f9` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.210437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f5a69032ab45babbc599aac755d2f9)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.212405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.254796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa81f90-4027-42cb-8200-47c385e74c3d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.259473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.260645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.260923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.261191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] PATH_INFO: `/v3/users/9c55a3f8565b491e818fcd6942f433c1` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.262347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c55a3f8565b491e818fcd6942f433c1)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.264262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.272182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f5a69032ab45babbc599aac755d2f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.282867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f5a69032ab45babbc599aac755d2f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.284739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f5a69032ab45babbc599aac755d2f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.285479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f5a69032ab45babbc599aac755d2f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.307821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f5a69032ab45babbc599aac755d2f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.327987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c55a3f8565b491e818fcd6942f433c1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.331491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 902/1790] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/users/58f5a69032ab45babbc599aac755d2f9 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:50:58.332402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.334228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.339775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c55a3f8565b491e818fcd6942f433c1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.340861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c55a3f8565b491e818fcd6942f433c1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.341880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c55a3f8565b491e818fcd6942f433c1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.351027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-71ec3497-1483-4882-927b-4222aa5ce3dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c55a3f8565b491e818fcd6942f433c1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.369054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.373718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 889/1791] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/users/9c55a3f8565b491e818fcd6942f433c1 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.374936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.375220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.375499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.403230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:58.404478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.426150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de55ec8-a7c4-40d2-bdaf-fed3fe4b0649 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.455017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.477431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 890/1792] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:58.489102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.490548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.516476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.517118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.517401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.517674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] PATH_INFO: `/v3/projects/33b93841fe8b4aa29329f6c0e9e80ed2` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.518845 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=33b93841fe8b4aa29329f6c0e9e80ed2)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.520755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.575616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99bc37-a9f8-499e-8b80-e939902f203e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.590291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33b93841fe8b4aa29329f6c0e9e80ed2', 'request_id': 'req-b4d14ef4-3c52-4eda-b696-667dc3765a8f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.598078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33b93841fe8b4aa29329f6c0e9e80ed2', 'request_id': 'req-b4d14ef4-3c52-4eda-b696-667dc3765a8f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.609326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33b93841fe8b4aa29329f6c0e9e80ed2', 'request_id': 'req-b4d14ef4-3c52-4eda-b696-667dc3765a8f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.623408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] The token cache is being invalidate because project 33b93841fe8b4aa29329f6c0e9e80ed2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:58.623705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.624377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b4d14ef4-3c52-4eda-b696-667dc3765a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:58.629248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 903/1793] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/projects/33b93841fe8b4aa29329f6c0e9e80ed2 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:58.630931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16021468-1201-41c0-b928-ce93d27a5022 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.632078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16021468-1201-41c0-b928-ce93d27a5022 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.632078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16021468-1201-41c0-b928-ce93d27a5022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.639325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.640084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.640505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.640838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.643506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.653508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-16021468-1201-41c0-b928-ce93d27a5022 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:58.654727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-16021468-1201-41c0-b928-ce93d27a5022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.732123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.732796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 904/1794] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:58.733690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.748909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.749205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.749483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.752729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 891/1795] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:50:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:50:58.759441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.774127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:58.777005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.786237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 905/1796] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:58.790092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.803026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797aef3-461c-476a-846d-36f64f10a06e service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.808761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.809392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.809670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.809984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.810630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:58.810956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.812966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.830785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 892/1797] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:58.830975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.841624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0646c3-424d-4843-b847-406ffff62fce None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.847282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.847981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.848262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.848698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.849198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.852374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:58.885791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 906/1798] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:50:58.887818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.898743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6408d067-6939-4894-b743-322f00f1b48a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:58.905415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:58.909069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:58.970287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:58.990038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:58.992207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:58.992840 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:58.993178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:58.993926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:58.997056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.023392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e842a3-c7d0-470e-83c1-85f1e707a444 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.028397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 893/1799] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:58 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:59.029148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.029930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.030050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.030347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.030700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] PATH_INFO: `/v3/projects/c7b4109314cc48cea6a213ac1d6ea0f8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.032182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7b4109314cc48cea6a213ac1d6ea0f8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.035177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.042190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b5438a-794e-4b88-bd68-9503e3a5d71e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.047192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.047925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.048262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.048591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.049301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.051080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.064049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 894/1800] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:59.068177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.079581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fb8e1-1bc2-44e2-8dfd-f512c0fe827c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.085216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.085910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.086189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.086453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.087086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.089234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.103348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 895/1801] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:59.107192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7b4109314cc48cea6a213ac1d6ea0f8', 'request_id': 'req-80283367-7920-4297-acc4-a392dec4e8a4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.107677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.114700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7b4109314cc48cea6a213ac1d6ea0f8', 'request_id': 'req-80283367-7920-4297-acc4-a392dec4e8a4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.118981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01830114-88a0-4e5e-8326-ea086c83171a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.123170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7b4109314cc48cea6a213ac1d6ea0f8', 'request_id': 'req-80283367-7920-4297-acc4-a392dec4e8a4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.135730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] The token cache is being invalidate because project c7b4109314cc48cea6a213ac1d6ea0f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:59.136054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.136755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-80283367-7920-4297-acc4-a392dec4e8a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.138611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 907/1802] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:58 2022] DELETE /identity/v3/projects/c7b4109314cc48cea6a213ac1d6ea0f8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.142989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.143270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.143615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.165772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:59.166954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.198796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07099f-f1fe-492c-a269-ed5332fe245d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.199907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07099f-f1fe-492c-a269-ed5332fe245d admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.200369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07099f-f1fe-492c-a269-ed5332fe245d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.200625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07099f-f1fe-492c-a269-ed5332fe245d admin admin] PATH_INFO: `/v3/projects/765cc385a7c4472b854a9620e618ffbc/users/1c11bd424a494db7af1fa54dea4a5e91/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.225208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07099f-f1fe-492c-a269-ed5332fe245d admin admin] RBAC: Authorizing `identity:create_grant(project_id=765cc385a7c4472b854a9620e618ffbc, user_id=1c11bd424a494db7af1fa54dea4a5e91, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.227969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07099f-f1fe-492c-a269-ed5332fe245d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.234643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 908/1803] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:59.235478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.238634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.248947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 896/1804] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:59 2022] PUT /identity/v3/projects/765cc385a7c4472b854a9620e618ffbc/users/1c11bd424a494db7af1fa54dea4a5e91/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.249777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.250207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66f78-5888-4d26-bcfb-d5e2c184c6aa None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.250495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66f78-5888-4d26-bcfb-d5e2c184c6aa None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.250769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e66f78-5888-4d26-bcfb-d5e2c184c6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.263024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f8c585-3cec-4742-aa07-7807a5a61cf0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.277647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c1e66f78-5888-4d26-bcfb-d5e2c184c6aa None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:59.279073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c1e66f78-5888-4d26-bcfb-d5e2c184c6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.325884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.326529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.326812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.327085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] PATH_INFO: `/v3/projects/89f4c563630d406ebae7aedc3d2a9ac7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.328297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89f4c563630d406ebae7aedc3d2a9ac7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.330232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.354561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 897/1805] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:59.354561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.354844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.355073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.388399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89f4c563630d406ebae7aedc3d2a9ac7', 'request_id': 'req-91ec3816-d9ed-4e83-a68d-ba385bd7d614'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.389031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] MFA Rules not processed for user `1c11bd424a494db7af1fa54dea4a5e91`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:50:59.391864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.395004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89f4c563630d406ebae7aedc3d2a9ac7', 'request_id': 'req-91ec3816-d9ed-4e83-a68d-ba385bd7d614'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.404165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89f4c563630d406ebae7aedc3d2a9ac7', 'request_id': 'req-91ec3816-d9ed-4e83-a68d-ba385bd7d614'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.418257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] The token cache is being invalidate because project 89f4c563630d406ebae7aedc3d2a9ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:50:59.418645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.419400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.421148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 909/1806] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:59 2022] DELETE /identity/v3/projects/89f4c563630d406ebae7aedc3d2a9ac7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.421591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.422448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.426936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.467384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 898/1807] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:50:59.468680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.470289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.497018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec3816-d9ed-4e83-a68d-ba385bd7d614 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.502358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.503211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.503565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.503883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.504562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:50:59.505033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.506519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.520061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 910/1808] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:50:59.520887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.522753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.537107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd41d22-c0c3-47cb-80d8-86a5e3a2a13f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.543886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.544753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.545057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.545458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.546179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.548080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.552339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fdb79-ada4-403a-b350-7d1d5aa35632 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.557989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.558839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.559406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.559779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] PATH_INFO: `/v3/users/1b965ec2665e44ae99af89f423b303d7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.561039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b965ec2665e44ae99af89f423b303d7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.563098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.579783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 911/1809] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:59.580445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.594037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.621993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b965ec2665e44ae99af89f423b303d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.630209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b965ec2665e44ae99af89f423b303d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.630829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b965ec2665e44ae99af89f423b303d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.631443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b965ec2665e44ae99af89f423b303d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.639558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b965ec2665e44ae99af89f423b303d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:50:59.653411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811496f-d6ae-490b-8366-da42f75d5e45 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.657618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 899/1810] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:50:59 2022] DELETE /identity/v3/users/1b965ec2665e44ae99af89f423b303d7 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.658496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.659801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.661439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.661944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.662251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.662521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.663152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.665039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.701308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 912/1811] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/projects => generated 399 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:59.702294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.704127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:50:59.715881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:50:59.743330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e132071e-23c9-47c3-b93a-47c50d0c8665 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.788889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2529eb-1c9b-4ef5-b6d1-79aed701560c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.794011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.794619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.795045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.795884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.796003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.797937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.816655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.820455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.820455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.820455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.820455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.823155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.840176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 913/1812] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:59.844682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.856444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac60afd-f1cc-4f58-af31-01839d16361d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.862835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.863821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.864055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.864390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.864666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 900/1813] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:50:59 2022] POST /identity/v3/users => generated 300 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:50:59.865123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.867865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.868553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.880264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 914/1814] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:50:59.880589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb8556-f872-452f-8b88-56cdd7499c7c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.884290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.885869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.886537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.886766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.887040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.887685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.889277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.894936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c722757-83ec-4a1f-9c3a-9f33953ba619 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.900897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.901511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.901814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.902086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.902979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.905116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.905314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 901/1815] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:59.906277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.921029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 915/1816] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:50:59.921957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.923905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946507d2-ec86-46a6-86f5-b7040e193d49 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.934016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a64273-4aa4-4394-851d-dd52c8b188d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.934219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7e0607-2a69-4a29-9d84-51bf180c83ed admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:50:59.934694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a64273-4aa4-4394-851d-dd52c8b188d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.935086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a64273-4aa4-4394-851d-dd52c8b188d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.935359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a64273-4aa4-4394-851d-dd52c8b188d0 admin admin] PATH_INFO: `/v3/projects/42e8535a6f564e13a5eb79c307f5b32e/users/b70b5de20ce645e3a7cbe43ba89bb302/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.939812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:50:59.940475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:50:59.940755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:50:59.941335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] PATH_INFO: `/v3/projects/5e3729884d6141bd893c33c401cf9fbd/users/c8dcfd2be7764f0d901bf503239502ab/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:50:59.960824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a64273-4aa4-4394-851d-dd52c8b188d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e8535a6f564e13a5eb79c307f5b32e, user_id=b70b5de20ce645e3a7cbe43ba89bb302, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.961563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e3729884d6141bd893c33c401cf9fbd, user_id=c8dcfd2be7764f0d901bf503239502ab, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:50:59.966886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.973878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a64273-4aa4-4394-851d-dd52c8b188d0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:50:59.985519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 916/1817] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:59 2022] PUT /identity/v3/projects/5e3729884d6141bd893c33c401cf9fbd/users/c8dcfd2be7764f0d901bf503239502ab/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:50:59.992787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:50:59.995558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.003919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 902/1818] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:50:59 2022] PUT /identity/v3/projects/42e8535a6f564e13a5eb79c307f5b32e/users/b70b5de20ce645e3a7cbe43ba89bb302/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:00.005988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.006340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.006668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.049545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] MFA Rules not processed for user `c8dcfd2be7764f0d901bf503239502ab`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.050818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.062999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.096468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c314e692-535b-4b8b-971d-05c4d36eaa20 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.127586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 903/1819] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.128435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.168722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.169498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.169613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.169908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] PATH_INFO: `/v3/projects/163dc1f58e304399b4cf295b7c0dcc45` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.171620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] RBAC: Authorizing `identity:delete_project(project_id=163dc1f58e304399b4cf295b7c0dcc45)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.173990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.176959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.217242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8cea7-73d3-46a5-8050-0a61df5e181c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.225931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '163dc1f58e304399b4cf295b7c0dcc45', 'request_id': 'req-cb4ad0fe-51b3-486d-aba2-92f4612f719d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.232924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '163dc1f58e304399b4cf295b7c0dcc45', 'request_id': 'req-cb4ad0fe-51b3-486d-aba2-92f4612f719d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.240050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '163dc1f58e304399b4cf295b7c0dcc45', 'request_id': 'req-cb4ad0fe-51b3-486d-aba2-92f4612f719d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.251213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] The token cache is being invalidate because project 163dc1f58e304399b4cf295b7c0dcc45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:00.251510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.252154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.253653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 917/1820] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:50:59 2022] DELETE /identity/v3/projects/163dc1f58e304399b4cf295b7c0dcc45 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:00.254493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.255808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.274562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d30465-b692-4393-912f-a375060fa32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.275169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d30465-b692-4393-912f-a375060fa32c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.275967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d30465-b692-4393-912f-a375060fa32c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.276236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d30465-b692-4393-912f-a375060fa32c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.276875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d30465-b692-4393-912f-a375060fa32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.278496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d30465-b692-4393-912f-a375060fa32c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.288618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 904/1821] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:00.289809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.290090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.290363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.309668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.310916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.319039 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 905/1822] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.326501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.326501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.326687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ad0fe-51b3-486d-aba2-92f4612f719d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.328462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbc41c-235b-45c2-afb2-e6a8af7d00ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.329069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbc41c-235b-45c2-afb2-e6a8af7d00ab admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.329350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbc41c-235b-45c2-afb2-e6a8af7d00ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.329649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbc41c-235b-45c2-afb2-e6a8af7d00ab admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.330316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbc41c-235b-45c2-afb2-e6a8af7d00ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.332259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbc41c-235b-45c2-afb2-e6a8af7d00ab admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.332769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0a6cb-d0a1-47c1-9737-1002887b3777 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.337542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29065361-a11f-4d07-9d33-a7dd53147b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.338732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29065361-a11f-4d07-9d33-a7dd53147b3d admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.339004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29065361-a11f-4d07-9d33-a7dd53147b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.339274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29065361-a11f-4d07-9d33-a7dd53147b3d admin admin] PATH_INFO: `/v3/projects/42e8535a6f564e13a5eb79c307f5b32e/users/b70b5de20ce645e3a7cbe43ba89bb302/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.341204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29065361-a11f-4d07-9d33-a7dd53147b3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=42e8535a6f564e13a5eb79c307f5b32e, user_id=b70b5de20ce645e3a7cbe43ba89bb302, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.343868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29065361-a11f-4d07-9d33-a7dd53147b3d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.354691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.355711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 906/1823] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:00 2022] PUT /identity/v3/projects/42e8535a6f564e13a5eb79c307f5b32e/users/b70b5de20ce645e3a7cbe43ba89bb302/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:00.357133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.357424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.357698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.364419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 918/1824] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/projects => generated 399 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:00.368804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.369126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.369399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.377409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.378645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.396932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] MFA Rules not processed for user `b70b5de20ce645e3a7cbe43ba89bb302`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:00.398147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.440571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 907/1825] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.441310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.459332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a83ac-01d6-4ee7-a7ab-d31e13d82867 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.469124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.470000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.470338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.470676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.471379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.473531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.483339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 919/1826] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:00.483339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.491439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2a377-fde9-4a9d-8f0d-fe539b81ab50 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.496979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.497789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.498141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.498509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.499260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:00.499574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.501227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.508499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 908/1827] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/users => generated 306 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:00.509649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.511785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:00.513706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 920/1828] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:00.514616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.527983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0089f-2519-4423-960c-f84ac6af5941 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.536684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.537442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.537910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.538372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.539273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.541648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.557352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 921/1829] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:00.557352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.566489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb84c139-7896-4bed-bc13-bf0685d98f82 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.573168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.573839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.573972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.574123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.574400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.575040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.576944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.587610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8c9aad-41c2-4199-a88d-f42ee8100b70 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.592723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.594150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.594573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.594854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.606002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.607421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.614900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 922/1830] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:00.614900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.630553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 909/1831] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:00.641476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.656932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004ddcd5-f2fe-499d-96f1-b9ad3e69e45a service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.699253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6791b016-e66e-41a1-881b-f767aef92cd6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.704549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.705452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.706119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.706407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] PATH_INFO: `/v3/projects/c8117381b3ac4a0f9f3986b33caa32fb/users/e7f1b53fc8394c92af0df4b0b7c9b203/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.728909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.729604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.729977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.730315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.731019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.733358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.747079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8117381b3ac4a0f9f3986b33caa32fb, user_id=e7f1b53fc8394c92af0df4b0b7c9b203, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.749972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.764203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 910/1832] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:00 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:00.768344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.769768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 923/1833] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:00 2022] PUT /identity/v3/projects/c8117381b3ac4a0f9f3986b33caa32fb/users/e7f1b53fc8394c92af0df4b0b7c9b203/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:00.774002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.786991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba12a44-f367-4c77-ab6f-383c7d8af981 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.798719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:00.857437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05833d06-6dd4-443b-adb2-35aee483c6d2 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.864395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.865027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.865875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.866063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.866322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.866486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.866717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.866997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.867494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.868342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.869286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.872350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.883335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 911/1834] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:00.883335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.884389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 924/1835] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:00.885473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.892799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56ec44-6dd3-4241-b35c-2a9c9e61ced8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.895740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdcf958-f5b5-43e2-b57c-3d184b50e8a9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.897565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.898228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.898538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.898770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] PATH_INFO: `/v3/users/1c11bd424a494db7af1fa54dea4a5e91` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.899950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c11bd424a494db7af1fa54dea4a5e91)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.901983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.902544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.902753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.903106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.903451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.904164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.905833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.916716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 925/1836] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:00.917638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.927585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d80ae6-db43-444d-92d0-ee089c2a3f94 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:00.933839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:00.934608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:00.934967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:00.935335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] PATH_INFO: `/v3/projects/c8117381b3ac4a0f9f3986b33caa32fb/users/e7f1b53fc8394c92af0df4b0b7c9b203/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:00.938930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8117381b3ac4a0f9f3986b33caa32fb, user_id=e7f1b53fc8394c92af0df4b0b7c9b203, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:00.942173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:00.965045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 926/1837] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:00 2022] PUT /identity/v3/projects/c8117381b3ac4a0f9f3986b33caa32fb/users/e7f1b53fc8394c92af0df4b0b7c9b203/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:00.965045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:00.967541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c11bd424a494db7af1fa54dea4a5e91'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.978277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c11bd424a494db7af1fa54dea4a5e91'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.978864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c11bd424a494db7af1fa54dea4a5e91'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.979486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c11bd424a494db7af1fa54dea4a5e91'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.988301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-97e42632-78e1-43b6-a603-fc69c4016c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c11bd424a494db7af1fa54dea4a5e91'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:00.990268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.012470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 912/1838] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:00 2022] DELETE /identity/v3/users/1c11bd424a494db7af1fa54dea4a5e91 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:01.013952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68961e6-3869-433b-a26e-f7cfbb714ffb None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.014336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68961e6-3869-433b-a26e-f7cfbb714ffb None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.014707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68961e6-3869-433b-a26e-f7cfbb714ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.029088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250920f5-e4ab-4a0a-9860-b74897b05d07 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.045978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a68961e6-3869-433b-a26e-f7cfbb714ffb None None] MFA Rules not processed for user `e7f1b53fc8394c92af0df4b0b7c9b203`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:01.047407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a68961e6-3869-433b-a26e-f7cfbb714ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.094121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2f44bc-6688-480f-bcc8-3a0d949c9a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.095025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f44bc-6688-480f-bcc8-3a0d949c9a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.095380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f44bc-6688-480f-bcc8-3a0d949c9a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.095788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f44bc-6688-480f-bcc8-3a0d949c9a0c admin admin] PATH_INFO: `/v3/projects/dba514425319450f9606867f4bd48664/users/50cd0f5e6a2e426db9901e6f47693338/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.115465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f44bc-6688-480f-bcc8-3a0d949c9a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dba514425319450f9606867f4bd48664, user_id=50cd0f5e6a2e426db9901e6f47693338, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.118325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2f44bc-6688-480f-bcc8-3a0d949c9a0c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.137365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 927/1839] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:00 2022] PUT /identity/v3/projects/dba514425319450f9606867f4bd48664/users/50cd0f5e6a2e426db9901e6f47693338/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:01.140845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 913/1840] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:01.141066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.141066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.141066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.142402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.142692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.142969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.164488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:01.167951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.174155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:01.175404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.192572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.255782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 914/1841] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:01.256028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 928/1842] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:01.261857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.263313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.271907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.274941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.343761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc2243-c640-4de0-bd73-d27dad17268d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.347808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.348717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.348996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.349264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.355720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b85fb1-8c24-4ea1-ace6-4f96e1b11853 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.360164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.360785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.361158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.361925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.374484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.376449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.379564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:01.393676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 929/1843] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:01.394812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.403783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cdd4c0-f241-4046-b2fc-2c851c0dadc9 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.408025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.408628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.408904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.409301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.435307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.435307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.438132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 930/1844] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:01.441399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.443203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.456054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 915/1845] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:01.536059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:01.548442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa7ebc-b8dd-42a6-8f11-28dd773cf49e service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:01.552818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:01.553648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:01.554042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:01.554410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:01.557169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:01.650363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:01.652333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:01.678987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 931/1846] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:01 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:02.218730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.220649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.252085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9d5c6-318f-48fd-967c-70032e637f9f service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.283977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.288433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.288812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.289105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] PATH_INFO: `/v3/projects/765cc385a7c4472b854a9620e618ffbc` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.295402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=765cc385a7c4472b854a9620e618ffbc)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.299241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.378006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '765cc385a7c4472b854a9620e618ffbc', 'request_id': 'req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.386554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '765cc385a7c4472b854a9620e618ffbc', 'request_id': 'req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.396001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '765cc385a7c4472b854a9620e618ffbc', 'request_id': 'req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.413850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] The token cache is being invalidate because project 765cc385a7c4472b854a9620e618ffbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:02.414713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.415590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-10a906d6-c0ea-4b86-9ea8-4b4f7ee4de4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.417870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 916/1847] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:02 2022] DELETE /identity/v3/projects/765cc385a7c4472b854a9620e618ffbc => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:02.422048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.437303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e634e3-b858-432f-a150-e5c343fec819 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.505407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:02.523078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.523732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.524015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.524290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] PATH_INFO: `/v3/users/b70b5de20ce645e3a7cbe43ba89bb302` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.526380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b70b5de20ce645e3a7cbe43ba89bb302)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.528395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.553117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.553415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.553711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.613751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] MFA Rules not processed for user `50cd0f5e6a2e426db9901e6f47693338`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:02.615958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.653693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b70b5de20ce645e3a7cbe43ba89bb302'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.679387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b70b5de20ce645e3a7cbe43ba89bb302'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.680061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b70b5de20ce645e3a7cbe43ba89bb302'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.680751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b70b5de20ce645e3a7cbe43ba89bb302'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.694780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b70b5de20ce645e3a7cbe43ba89bb302'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:02.736589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 932/1848] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:02 2022] DELETE /identity/v3/users/b70b5de20ce645e3a7cbe43ba89bb302 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:02.737219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 917/1849] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:02.741579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.743024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:02.820496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e651b767-4bc9-459d-a134-fe7f4db40663 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.826781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.827398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.827678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.827952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.828601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.830766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.860274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 933/1850] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:02 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:02.864442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.875641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f55276-f7fd-4acf-a6e9-8cce1d6cedfc None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.907974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:02.943157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.943766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.944042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.944312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.944943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.946886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:02.971313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 918/1851] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:02 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:02.972690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:02.983478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c38a67-8e72-4ecd-86e1-8b1a31b252aa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:02.988145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:02.988685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:02.988963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:02.989233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:02.989912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:02.991521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.002519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 934/1852] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:03.006825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.020622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd34f3b7-6a06-4b8d-9bd1-3c7018c6fb57 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.026350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.027020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.027349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.027618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] PATH_INFO: `/v3/projects/dcca8cfbbef84a618925a96ef5d75478/users/707a84a0ad384c7bb40deccfbe07b4bb/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.035733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.037529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.046644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcca8cfbbef84a618925a96ef5d75478, user_id=707a84a0ad384c7bb40deccfbe07b4bb, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.050522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.051182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a015c096-58f0-4087-ba5c-a935051e6853 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.057828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.058802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.059359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.059651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] PATH_INFO: `/v3/projects/42e8535a6f564e13a5eb79c307f5b32e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.061510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] RBAC: Authorizing `identity:delete_project(project_id=42e8535a6f564e13a5eb79c307f5b32e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.063623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.070434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 919/1853] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:03 2022] PUT /identity/v3/projects/dcca8cfbbef84a618925a96ef5d75478/users/707a84a0ad384c7bb40deccfbe07b4bb/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:03.074909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.114331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:03.127673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42e8535a6f564e13a5eb79c307f5b32e', 'request_id': 'req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.139201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42e8535a6f564e13a5eb79c307f5b32e', 'request_id': 'req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.143389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba9e525-1175-4945-a3eb-13adf0e09cc9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.150519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42e8535a6f564e13a5eb79c307f5b32e', 'request_id': 'req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.168357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] The token cache is being invalidate because project 42e8535a6f564e13a5eb79c307f5b32e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:03.168649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.169801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2c696a9b-dfd5-4ca1-84a0-357a9ebd195c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:03.171926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 935/1854] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:03 2022] DELETE /identity/v3/projects/42e8535a6f564e13a5eb79c307f5b32e => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:03.176321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8b2c6-6951-465f-90ee-8e684b070005 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.176734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8b2c6-6951-465f-90ee-8e684b070005 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.177066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a8b2c6-6951-465f-90ee-8e684b070005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.198880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-63a8b2c6-6951-465f-90ee-8e684b070005 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:03.200118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-63a8b2c6-6951-465f-90ee-8e684b070005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.220247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.220999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.221220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.221640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.222329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.224716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.237660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 920/1855] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:03.241981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.244126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.260574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ac5ba-72a3-48b4-b48b-78f4ec6f2798 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.265749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.266584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.266890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.267160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] PATH_INFO: `/v3/projects/dcca8cfbbef84a618925a96ef5d75478/users/707a84a0ad384c7bb40deccfbe07b4bb/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.269121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcca8cfbbef84a618925a96ef5d75478, user_id=707a84a0ad384c7bb40deccfbe07b4bb, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.274035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.285934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 936/1856] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:03.298392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592663f0-609b-4656-99d1-0e76f438de15 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.298892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592663f0-609b-4656-99d1-0e76f438de15 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.299164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592663f0-609b-4656-99d1-0e76f438de15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.303831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 921/1857] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:03 2022] PUT /identity/v3/projects/dcca8cfbbef84a618925a96ef5d75478/users/707a84a0ad384c7bb40deccfbe07b4bb/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:03.348276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-592663f0-609b-4656-99d1-0e76f438de15 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:03.349807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-592663f0-609b-4656-99d1-0e76f438de15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.460360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 937/1858] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:03.465961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.468739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db71a-fcbf-4f43-bb14-7f618c4216c8 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.469138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db71a-fcbf-4f43-bb14-7f618c4216c8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.469373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682db71a-fcbf-4f43-bb14-7f618c4216c8 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.471482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 938/1859] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:51:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:03.480583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effce311-cb72-4851-b074-baea0417d7fe None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.485858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effce311-cb72-4851-b074-baea0417d7fe None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.486020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6bb5e-94d2-442e-bda4-73ce1cc83675 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.486292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effce311-cb72-4851-b074-baea0417d7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.494673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.495316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.495749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.496287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.497063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:03.497508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.499271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.521003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-effce311-cb72-4851-b074-baea0417d7fe None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:03.523047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-effce311-cb72-4851-b074-baea0417d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.528377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 922/1860] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:03.529690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.545365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d206ffd4-9d6d-4989-aba7-f3b47ab4df85 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.552415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.553834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.554621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.554805 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.555685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.559059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.589196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:03.619253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:03.641814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 923/1861] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:03.642907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.646603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 939/1862] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:03.713219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effce311-cb72-4851-b074-baea0417d7fe None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.714858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-effce311-cb72-4851-b074-baea0417d7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:03.738843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-effce311-cb72-4851-b074-baea0417d7fe None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.770584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea6db6-2cd6-414d-a5a7-0901dd14cadc admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.782110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.783448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.783997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.784460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.785467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.793941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.831834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 924/1863] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:03 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:03.836534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.867088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dee6123-c679-4f11-96a3-78674029f385 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.869857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.871268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.871550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.871847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.879076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.880255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.880369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.880576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.881241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.885584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.889557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.892478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.924438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 940/1864] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:03.927073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 925/1865] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:03.932025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.945468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3733e9e9-0af0-4e84-9c6d-e0a9d2c6d964 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:03.953149 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021f00d-daac-4980-8a5c-3242ede4e5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:03.953806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f00d-daac-4980-8a5c-3242ede4e5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:03.954124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f00d-daac-4980-8a5c-3242ede4e5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:03.954429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f00d-daac-4980-8a5c-3242ede4e5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:03.955309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021f00d-daac-4980-8a5c-3242ede4e5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:03.957056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b021f00d-daac-4980-8a5c-3242ede4e5a8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:03.971223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 941/1866] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:03.978647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:03.995170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6acd183-b84c-4add-a9c6-4913fb80941f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.001198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.001963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.002250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.002833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] PATH_INFO: `/v3/projects/2ab9e5a005ac45daa36332ca7f9d18f5/users/7f8f17095cba4819900bcf96f1696210/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.024417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ab9e5a005ac45daa36332ca7f9d18f5, user_id=7f8f17095cba4819900bcf96f1696210, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.028127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.062806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 926/1867] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:03 2022] PUT /identity/v3/projects/2ab9e5a005ac45daa36332ca7f9d18f5/users/7f8f17095cba4819900bcf96f1696210/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:04.074025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.074524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.074806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.115770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] MFA Rules not processed for user `7f8f17095cba4819900bcf96f1696210`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:04.117302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.218828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.224842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 942/1868] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:04.342899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395f4e9e-c09d-4ad8-a058-b027f327446e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.359565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.360214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.360491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.360856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.361412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.363463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.394650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 927/1869] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:04.397514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.413292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f1ddc-aac0-48fd-b0ab-6f33137c22e2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.533955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193a48b-b762-4d1e-83a2-8149fb970d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.534784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a48b-b762-4d1e-83a2-8149fb970d27 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.534913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a48b-b762-4d1e-83a2-8149fb970d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.535178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193a48b-b762-4d1e-83a2-8149fb970d27 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.535861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a48b-b762-4d1e-83a2-8149fb970d27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.537837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193a48b-b762-4d1e-83a2-8149fb970d27 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.550067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:04.571144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 943/1870] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:04.580120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.593210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8f932-d696-4132-a3a9-2f15a03b464d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.593558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9033eb07-7b03-4c77-b26b-bf82eb85a35b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.593678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8f932-d696-4132-a3a9-2f15a03b464d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.594128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a8f932-d696-4132-a3a9-2f15a03b464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.599969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.600895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.600895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.601124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.601810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.604192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.621092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 928/1871] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:04.630404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.632610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e6a8f932-d696-4132-a3a9-2f15a03b464d None None] MFA Rules not processed for user `707a84a0ad384c7bb40deccfbe07b4bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:04.634031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e6a8f932-d696-4132-a3a9-2f15a03b464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.642876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e6831-397c-43d3-b768-d9af6f44a8e6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.651264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.652381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.653777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.654352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.655734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.658451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.684350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 929/1872] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:04.688409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.701303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818b71d-c5a1-46ef-83fd-6460cd7ede13 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.705688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.706445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.706759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.707033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] PATH_INFO: `/v3/projects/84b60bfeb66745f58081672693b540aa/users/f926f90bbebe46a79ab3b2350ed9f0a4/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.712261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:04.727898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b60bfeb66745f58081672693b540aa, user_id=f926f90bbebe46a79ab3b2350ed9f0a4, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:04.731236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:04.753489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 944/1874] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:04.753818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 930/1874] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:04 2022] PUT /identity/v3/projects/84b60bfeb66745f58081672693b540aa/users/f926f90bbebe46a79ab3b2350ed9f0a4/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:04.763783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.764156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.764738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.779979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:04.782966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.860975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] MFA Rules not processed for user `f926f90bbebe46a79ab3b2350ed9f0a4`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:04.863452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:04.938078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51913349-07b2-49f3-9dde-3ed069ea9516 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:04.946547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:04.947414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:04.947705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:04.947978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:04.957991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:05.065813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 945/1875] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:05.067169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.110735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.112216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.136411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 931/1876] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:05.209531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad4fcc-6f0a-4a54-8e8a-78d1d7708ab2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.225594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.227018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.227407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.227864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.229631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.233926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.263186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.297045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 946/1877] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:05.301536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.324674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4d0c9-b6a3-4234-8589-14754cc3c153 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.442234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.442896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.443147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.443416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.446283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.451266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.526425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 932/1878] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:05 2022] POST /identity/v3/users => generated 295 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:05.529403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.546381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871972a-9c02-46ed-8905-8d7bf92b554c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.569134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.569781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.570063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.570330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.570979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.573991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.627185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 947/1879] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:05.629179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.651739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc31bd9-a338-493b-8f4a-88c6bf3a6d9f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.674656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.676370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.676673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.677078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] PATH_INFO: `/v3/projects/7cc8d4b46aaf423b93bbceb9f74aa925/users/93a05480d5ef4d5e902bdf9316871e2f/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.732182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc8d4b46aaf423b93bbceb9f74aa925, user_id=93a05480d5ef4d5e902bdf9316871e2f, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.739991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.777107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 933/1880] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:05 2022] PUT /identity/v3/projects/7cc8d4b46aaf423b93bbceb9f74aa925/users/93a05480d5ef4d5e902bdf9316871e2f/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:05.779204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.795835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0046e5-8417-4452-891b-b8516797360f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:05.944865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6291246-2086-48a7-b37e-5841d2ad14b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:05.945684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6291246-2086-48a7-b37e-5841d2ad14b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:05.946363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6291246-2086-48a7-b37e-5841d2ad14b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:05.946847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6291246-2086-48a7-b37e-5841d2ad14b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:05.947680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6291246-2086-48a7-b37e-5841d2ad14b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:05.951155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6291246-2086-48a7-b37e-5841d2ad14b6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:05.970446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 948/1881] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:05 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:05.974698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:05.986068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:05.992635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930068b2-6f65-4b7c-b593-67f5e46c3193 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.004379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.005055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.005329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.005596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] PATH_INFO: `/v3/projects/7cc8d4b46aaf423b93bbceb9f74aa925/users/93a05480d5ef4d5e902bdf9316871e2f/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.008425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cc8d4b46aaf423b93bbceb9f74aa925, user_id=93a05480d5ef4d5e902bdf9316871e2f, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.013178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.051389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 934/1882] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:05 2022] PUT /identity/v3/projects/7cc8d4b46aaf423b93bbceb9f74aa925/users/93a05480d5ef4d5e902bdf9316871e2f/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:06.061983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.062519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.062793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.109297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] MFA Rules not processed for user `93a05480d5ef4d5e902bdf9316871e2f`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:06.110750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.256248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 949/1883] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:06.271473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.273768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.374417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47113e3d-f12d-4112-94a4-13523dacbf68 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.378285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.379951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.380305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.380652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.399488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.401480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.413663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 935/1884] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:06.465004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.475339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3356c-dd7c-4a72-8cef-f299ca8e4da6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.480504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.480504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.480880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.481253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.615043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.616501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.643307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 950/1885] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:06.831349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.831349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:06.841756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:06.846718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:06.853930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fc9c4-43c2-41d6-a518-22cf9001b26d service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.859038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0994f491-ccea-4431-95b9-9a45e21710f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.859978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994f491-ccea-4431-95b9-9a45e21710f1 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.860327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994f491-ccea-4431-95b9-9a45e21710f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.860673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0994f491-ccea-4431-95b9-9a45e21710f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.926543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5089c8-eeb7-4078-9307-03f72f654f00 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:06.935265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:06.936127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:06.936530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:06.936818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] PATH_INFO: `/v3/users/c8dcfd2be7764f0d901bf503239502ab` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:06.938558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8dcfd2be7764f0d901bf503239502ab)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.942170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.942726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994f491-ccea-4431-95b9-9a45e21710f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:06.949463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0994f491-ccea-4431-95b9-9a45e21710f1 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:06.964692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 951/1886] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:06 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:07.006935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8dcfd2be7764f0d901bf503239502ab'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.019296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8dcfd2be7764f0d901bf503239502ab'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.020336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8dcfd2be7764f0d901bf503239502ab'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.020883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8dcfd2be7764f0d901bf503239502ab'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.034945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8dcfd2be7764f0d901bf503239502ab'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.066417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 936/1887] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:06 2022] DELETE /identity/v3/users/c8dcfd2be7764f0d901bf503239502ab => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:07.073031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.073369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.073776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.102874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:07.102874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.225551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 952/1888] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:07.295041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:07.297833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.332482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:07.430226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60ad74-d5d5-42f0-a165-949d323c8b0b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:07.435271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:07.436099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.436535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.437286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.454994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:07.456926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:07.471814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 937/1889] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:07.861509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:07.863114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:07.875505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30b9eeb-f5bd-4c44-96d9-0423786be304 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:07.880962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:07.881613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:07.881918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:07.882191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] PATH_INFO: `/v3/projects/5e3729884d6141bd893c33c401cf9fbd` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:07.883349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e3729884d6141bd893c33c401cf9fbd)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:07.885337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:07.936926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3729884d6141bd893c33c401cf9fbd', 'request_id': 'req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.943556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3729884d6141bd893c33c401cf9fbd', 'request_id': 'req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.951629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3729884d6141bd893c33c401cf9fbd', 'request_id': 'req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.963510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] The token cache is being invalidate because project 5e3729884d6141bd893c33c401cf9fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:07.963801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.964452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:07.965965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 953/1890] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:07 2022] DELETE /identity/v3/projects/5e3729884d6141bd893c33c401cf9fbd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:07.970052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:07.980929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db16a5f0-9b56-4a5b-b845-10e8e6e48685 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.041666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.042325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.042599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.042870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] PATH_INFO: `/v3/users/e7f1b53fc8394c92af0df4b0b7c9b203` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.043981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7f1b53fc8394c92af0df4b0b7c9b203)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.045955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.091190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7f1b53fc8394c92af0df4b0b7c9b203'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.098074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7f1b53fc8394c92af0df4b0b7c9b203'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.098689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7f1b53fc8394c92af0df4b0b7c9b203'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.099292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7f1b53fc8394c92af0df4b0b7c9b203'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.106149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8ac4dd2e-c090-47c5-bae9-b76b2d28cc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7f1b53fc8394c92af0df4b0b7c9b203'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.122982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 938/1891] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:07 2022] DELETE /identity/v3/users/e7f1b53fc8394c92af0df4b0b7c9b203 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:08.428272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.430036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.463999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1062d-0d3c-4ac2-bd7b-58ae95cfbeaa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.509897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:08.569987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.570662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.570937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.571212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] PATH_INFO: `/v3/projects/c8117381b3ac4a0f9f3986b33caa32fb` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.573553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8117381b3ac4a0f9f3986b33caa32fb)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.576338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.630700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8117381b3ac4a0f9f3986b33caa32fb', 'request_id': 'req-502ba11a-d8b6-487e-864e-6aac742851fe'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.639181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8117381b3ac4a0f9f3986b33caa32fb', 'request_id': 'req-502ba11a-d8b6-487e-864e-6aac742851fe'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.647520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8117381b3ac4a0f9f3986b33caa32fb', 'request_id': 'req-502ba11a-d8b6-487e-864e-6aac742851fe'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.668384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] The token cache is being invalidate because project c8117381b3ac4a0f9f3986b33caa32fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:08.668662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.669535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-502ba11a-d8b6-487e-864e-6aac742851fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:08.671434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 954/1892] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:08 2022] DELETE /identity/v3/projects/c8117381b3ac4a0f9f3986b33caa32fb => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:08.676497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.676865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.677139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.702173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:08.704063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.745200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:08.837846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 939/1893] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:08.839790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.840145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.840547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.879565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:08.885405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:08.916248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 955/1894] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:08.921167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.937271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9767363-a145-40e4-b9fb-7a7d1bcf5a0d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:08.945544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:08.946515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:08.947336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:08.948102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:08.949358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:08.949891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:08.953081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:08.982048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 940/1895] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:08.985810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:08.996795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c678da81-f1c0-43ac-ae9f-659c5e23f6b5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.003253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.003874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.004184 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.004461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.005155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.007049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.046689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 956/1896] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:08 2022] POST /identity/v3/projects => generated 387 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:09.064890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.077152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b78b6a-f3b0-40bf-ab33-cd74c1a3abed admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.145939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.146553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.146834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.147109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.147754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.149715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.171149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 941/1897] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/users => generated 294 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:09.175166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.186167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee58ead-9357-4c22-bf6d-efb20fbe89d9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.190717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.191308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.191585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.191858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.192493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.194110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.203813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 957/1898] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:09.207847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.218345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8bc725-5793-4f75-a18f-36f94f83b5fc admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.222883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37e486f-37c1-45b6-a5f2-97836657199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.223472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e486f-37c1-45b6-a5f2-97836657199f admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.223750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e486f-37c1-45b6-a5f2-97836657199f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.224020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37e486f-37c1-45b6-a5f2-97836657199f admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.224686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e486f-37c1-45b6-a5f2-97836657199f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.226278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37e486f-37c1-45b6-a5f2-97836657199f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.237467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 942/1899] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:09.241542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.251258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb20e40-c5bd-43ca-a3d7-094a2c29358b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.255766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.256403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.256683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.256961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] PATH_INFO: `/v3/projects/d8756b98abc348b9b185fef8a50e6eb7/users/77da49a39a434d39867ea71b553fa432/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.271029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8756b98abc348b9b185fef8a50e6eb7, user_id=77da49a39a434d39867ea71b553fa432, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.273668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.286663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 958/1900] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:09 2022] PUT /identity/v3/projects/d8756b98abc348b9b185fef8a50e6eb7/users/77da49a39a434d39867ea71b553fa432/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:09.297645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.297974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.298253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.325265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] MFA Rules not processed for user `77da49a39a434d39867ea71b553fa432`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:09.326930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.389820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 943/1901] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:09.394162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.441055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:09.457359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acefeda2-d275-461f-8307-c8e902b1d468 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.461954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.463081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.463357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.463624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.464248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.466150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.492035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 959/1902] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:09.496040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.506636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e968c-cd63-451b-a907-868fb187ffb6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.538380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:09.576790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.577410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.577684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.578000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.578649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.580549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.603395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 944/1903] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/users => generated 300 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:09.607280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.617346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9355f-cabb-4fc1-b70c-4626b70ab312 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.622216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.622835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.623105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.623375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.624022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.625623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.638182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 960/1904] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:09.642224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.652627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99edcc45-769b-48a2-aa90-f8193aa266a3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.657678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.658350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.658623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.658896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] PATH_INFO: `/v3/projects/1050c9b619e14795973ec4253e0d2630/users/def11592e26748b3a59123254620b11a/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.675201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1050c9b619e14795973ec4253e0d2630, user_id=def11592e26748b3a59123254620b11a, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.677874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.692867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 945/1905] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:09 2022] PUT /identity/v3/projects/1050c9b619e14795973ec4253e0d2630/users/def11592e26748b3a59123254620b11a/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:09.697391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.708309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f924c-9dd2-4e25-a5a1-c107f0713e01 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.774379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa276b-e245-4417-a79a-674c3b7ad3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.775568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa276b-e245-4417-a79a-674c3b7ad3cc admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.775843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa276b-e245-4417-a79a-674c3b7ad3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.776110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa276b-e245-4417-a79a-674c3b7ad3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.776774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa276b-e245-4417-a79a-674c3b7ad3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.778438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39aa276b-e245-4417-a79a-674c3b7ad3cc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.788346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 961/1906] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:09.792327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.802397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2f90-49df-4fef-9965-53d2df67235c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:09.806876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:09.807509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.807779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.808045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] PATH_INFO: `/v3/projects/1050c9b619e14795973ec4253e0d2630/users/def11592e26748b3a59123254620b11a/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.809931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1050c9b619e14795973ec4253e0d2630, user_id=def11592e26748b3a59123254620b11a, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:09.812479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:09.822727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 946/1907] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:09 2022] PUT /identity/v3/projects/1050c9b619e14795973ec4253e0d2630/users/def11592e26748b3a59123254620b11a/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:09.834057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:09.834340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:09.834610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:09.867179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] MFA Rules not processed for user `def11592e26748b3a59123254620b11a`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:09.868382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:09.943429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 962/1908] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:09 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:09.951378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:09.953131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.014399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d45d2c-0d38-4c93-a0f1-b5332b4dd0e9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.018107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2f8fa-b694-4146-925f-18190d873ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.018707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2f8fa-b694-4146-925f-18190d873ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.018981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2f8fa-b694-4146-925f-18190d873ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.019251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2f8fa-b694-4146-925f-18190d873ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.029702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2f8fa-b694-4146-925f-18190d873ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.031615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2f8fa-b694-4146-925f-18190d873ec0 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.041922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 947/1909] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:09 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:10.252238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.253813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.373886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:10.428157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7419-fd7f-4f9d-9a0c-9f4b2a0c8de5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.436232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.437189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.441945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.442295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] PATH_INFO: `/v3/users/7f8f17095cba4819900bcf96f1696210` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.445343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f8f17095cba4819900bcf96f1696210)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.451317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.530773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f8f17095cba4819900bcf96f1696210'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:10.543195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f8f17095cba4819900bcf96f1696210'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:10.544460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f8f17095cba4819900bcf96f1696210'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:10.545308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f8f17095cba4819900bcf96f1696210'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:10.558562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f8f17095cba4819900bcf96f1696210'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:10.581365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 963/1910] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:10 2022] DELETE /identity/v3/users/7f8f17095cba4819900bcf96f1696210 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:10.586346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.586607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.586884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.609783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:10.611104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.710749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 948/1911] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:10.722072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.723775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:10.809516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b639a51-03ad-4ad3-a3d7-d39d34b4c371 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:10.813483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296143ab-9ca2-4dee-86ce-98715d0499ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:10.814191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296143ab-9ca2-4dee-86ce-98715d0499ea service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:10.814549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296143ab-9ca2-4dee-86ce-98715d0499ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:10.814821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296143ab-9ca2-4dee-86ce-98715d0499ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:10.845780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296143ab-9ca2-4dee-86ce-98715d0499ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:10.847606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296143ab-9ca2-4dee-86ce-98715d0499ea service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:10.861862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 964/1912] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:10.988321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:10.989804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.004114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af6ca5b-ee71-44fe-94e7-321b778ea11e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.009391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.010057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.010338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.010611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] PATH_INFO: `/v3/users/77da49a39a434d39867ea71b553fa432` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.012143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=77da49a39a434d39867ea71b553fa432)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.013679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.042113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:11.085610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '77da49a39a434d39867ea71b553fa432'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.099153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '77da49a39a434d39867ea71b553fa432'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.099780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '77da49a39a434d39867ea71b553fa432'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.100394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '77da49a39a434d39867ea71b553fa432'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.108276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '77da49a39a434d39867ea71b553fa432'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.126440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 949/1913] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:10 2022] DELETE /identity/v3/users/77da49a39a434d39867ea71b553fa432 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:11.130881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.131408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.131408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.151499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:11.152695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.212755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 965/1914] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:11.213286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.214865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.223268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.224708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.230670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafbfb29-5e92-444c-a435-e3b487079a5b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.231801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:11.236855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.237503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.237812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.238099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] PATH_INFO: `/v3/projects/2ab9e5a005ac45daa36332ca7f9d18f5` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.239545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ab9e5a005ac45daa36332ca7f9d18f5)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.241486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.291898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baae4b1-a8c2-4760-a2ac-bc5ef6117a3f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.301311 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ab9e5a005ac45daa36332ca7f9d18f5', 'request_id': 'req-31c9e164-20b2-4a1a-9a82-7574cdc10d93'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.309907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ab9e5a005ac45daa36332ca7f9d18f5', 'request_id': 'req-31c9e164-20b2-4a1a-9a82-7574cdc10d93'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.323272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ab9e5a005ac45daa36332ca7f9d18f5', 'request_id': 'req-31c9e164-20b2-4a1a-9a82-7574cdc10d93'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.340941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] The token cache is being invalidate because project 2ab9e5a005ac45daa36332ca7f9d18f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:11.341765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.342425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.345797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 950/1915] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:11 2022] DELETE /identity/v3/projects/2ab9e5a005ac45daa36332ca7f9d18f5 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:11.349782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.352448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.353406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.354738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.355110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.355519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.357360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:11.442415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.444042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.456415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 966/1916] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:11.461817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c9e164-20b2-4a1a-9a82-7574cdc10d93 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:11.466795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:11.467435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:11.467727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:11.468059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] PATH_INFO: `/v3/users/f926f90bbebe46a79ab3b2350ed9f0a4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:11.469130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f926f90bbebe46a79ab3b2350ed9f0a4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:11.472976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:11.498197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:11.530793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f926f90bbebe46a79ab3b2350ed9f0a4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.544977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f926f90bbebe46a79ab3b2350ed9f0a4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.545774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f926f90bbebe46a79ab3b2350ed9f0a4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.546411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f926f90bbebe46a79ab3b2350ed9f0a4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.582665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f926f90bbebe46a79ab3b2350ed9f0a4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:11.603257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 951/1917] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:11 2022] DELETE /identity/v3/users/f926f90bbebe46a79ab3b2350ed9f0a4 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:11.984207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:11.985649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.001826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77621b5-a1a1-4aac-98bf-0fecebf5f342 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.093382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.094163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.094503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.094843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] PATH_INFO: `/v3/projects/d8756b98abc348b9b185fef8a50e6eb7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.096762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8756b98abc348b9b185fef8a50e6eb7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.098303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:12.113591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:12.198324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:12.200180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.214646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8756b98abc348b9b185fef8a50e6eb7', 'request_id': 'req-8124a64a-e846-4c29-842d-96cefca46608'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.224023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fd928-9218-486a-8e95-32b6fe5015ab admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.245022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8756b98abc348b9b185fef8a50e6eb7', 'request_id': 'req-8124a64a-e846-4c29-842d-96cefca46608'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.259735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8756b98abc348b9b185fef8a50e6eb7', 'request_id': 'req-8124a64a-e846-4c29-842d-96cefca46608'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.278222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] The token cache is being invalidate because project d8756b98abc348b9b185fef8a50e6eb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:12.278521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.280430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.283618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 967/1918] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:11 2022] DELETE /identity/v3/projects/d8756b98abc348b9b185fef8a50e6eb7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:12.287692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:12.292988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.387774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.389186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.389471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.389784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] PATH_INFO: `/v3/projects/84b60bfeb66745f58081672693b540aa` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.391067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b60bfeb66745f58081672693b540aa)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.393411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:12.448177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8124a64a-e846-4c29-842d-96cefca46608 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.467866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.469305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.469673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.470083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] PATH_INFO: `/v3/users/def11592e26748b3a59123254620b11a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.471395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=def11592e26748b3a59123254620b11a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.473921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:12.479496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b60bfeb66745f58081672693b540aa', 'request_id': 'req-50153ae0-8d9f-45e9-87eb-a7382bf053c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.486501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b60bfeb66745f58081672693b540aa', 'request_id': 'req-50153ae0-8d9f-45e9-87eb-a7382bf053c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.498327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b60bfeb66745f58081672693b540aa', 'request_id': 'req-50153ae0-8d9f-45e9-87eb-a7382bf053c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.511257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] The token cache is being invalidate because project 84b60bfeb66745f58081672693b540aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:12.511542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.512212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.513978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 952/1919] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:12 2022] DELETE /identity/v3/projects/84b60bfeb66745f58081672693b540aa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:12.518532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:12.520822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:12.536188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def11592e26748b3a59123254620b11a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.552667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def11592e26748b3a59123254620b11a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.553317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def11592e26748b3a59123254620b11a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.554008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def11592e26748b3a59123254620b11a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.560470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:12.562898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def11592e26748b3a59123254620b11a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.588899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 968/1920] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:12 2022] DELETE /identity/v3/users/def11592e26748b3a59123254620b11a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:12.598217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50153ae0-8d9f-45e9-87eb-a7382bf053c0 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:12.694467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:12.696379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:12.697090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:12.697717 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] PATH_INFO: `/v3/users/93a05480d5ef4d5e902bdf9316871e2f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:12.699920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a05480d5ef4d5e902bdf9316871e2f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:12.702234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:12.788493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a05480d5ef4d5e902bdf9316871e2f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.805271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a05480d5ef4d5e902bdf9316871e2f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.806968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a05480d5ef4d5e902bdf9316871e2f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.807786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a05480d5ef4d5e902bdf9316871e2f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.819802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a05480d5ef4d5e902bdf9316871e2f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:12.867015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 953/1921] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:12 2022] DELETE /identity/v3/users/93a05480d5ef4d5e902bdf9316871e2f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:13.166338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.167904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.183296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02508e-1c35-4dbf-9c78-917041ee60aa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.206011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:13.289706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.291433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.291720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.292000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] PATH_INFO: `/v3/projects/1050c9b619e14795973ec4253e0d2630` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.293462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1050c9b619e14795973ec4253e0d2630)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.296242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.359899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.361371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.380113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1050c9b619e14795973ec4253e0d2630', 'request_id': 'req-e9f588ef-3f65-4404-902b-e783e3775366'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.382317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb150cb-3f2c-456a-86e0-a2ee39af1ea9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.388720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1050c9b619e14795973ec4253e0d2630', 'request_id': 'req-e9f588ef-3f65-4404-902b-e783e3775366'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.403078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1050c9b619e14795973ec4253e0d2630', 'request_id': 'req-e9f588ef-3f65-4404-902b-e783e3775366'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.431020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] The token cache is being invalidate because project 1050c9b619e14795973ec4253e0d2630 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:13.431020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.434947 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e9f588ef-3f65-4404-902b-e783e3775366 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.437817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 969/1922] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:13 2022] DELETE /identity/v3/projects/1050c9b619e14795973ec4253e0d2630 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:13.444290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f349cca-b8c8-4e07-b43e-2865106b6b55 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.445277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f349cca-b8c8-4e07-b43e-2865106b6b55 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.446279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f349cca-b8c8-4e07-b43e-2865106b6b55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.525928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1f349cca-b8c8-4e07-b43e-2865106b6b55 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:13.527714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1f349cca-b8c8-4e07-b43e-2865106b6b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.575117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:13.597958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.599430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.600030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.600584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] PATH_INFO: `/v3/projects/7cc8d4b46aaf423b93bbceb9f74aa925` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.602402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cc8d4b46aaf423b93bbceb9f74aa925)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.605166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.643704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 970/1923] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:13.652603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.652988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.653501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.685232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:13.687724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.701994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cc8d4b46aaf423b93bbceb9f74aa925', 'request_id': 'req-16c2757a-9c22-4819-8a69-b34af6966015'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.716668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cc8d4b46aaf423b93bbceb9f74aa925', 'request_id': 'req-16c2757a-9c22-4819-8a69-b34af6966015'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.739411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cc8d4b46aaf423b93bbceb9f74aa925', 'request_id': 'req-16c2757a-9c22-4819-8a69-b34af6966015'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.762667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:13.771897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] The token cache is being invalidate because project 7cc8d4b46aaf423b93bbceb9f74aa925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:13.772185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.772867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-16c2757a-9c22-4819-8a69-b34af6966015 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:13.779261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 954/1924] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:13 2022] DELETE /identity/v3/projects/7cc8d4b46aaf423b93bbceb9f74aa925 => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:13.780843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6807bb-c1d0-4758-9888-f912a686d171 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.781445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6807bb-c1d0-4758-9888-f912a686d171 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.781825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6807bb-c1d0-4758-9888-f912a686d171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.794641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 971/1925] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:13.798906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.817261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625d2ce0-85ce-4d5b-9fee-b8c47737da8b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.818546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0b6807bb-c1d0-4758-9888-f912a686d171 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:13.819994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0b6807bb-c1d0-4758-9888-f912a686d171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.832094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.832761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.833052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.833340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.838159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:13.838159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.838159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.848654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 955/1926] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:13.854846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 972/1927] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:13.858904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.859278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.859841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.860782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.874646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc82cdb-bb3c-4f69-979f-e720bf0e00fa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.881350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.882229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.882761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.883385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.884815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.884974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:13.886336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:13.887719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.899195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 956/1928] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:13.900309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.917073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc01b9d-95bd-4b9e-a15c-17dea5894837 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:13.926524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:13.927719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:13.928294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:13.928707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:13.929552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:13.930027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:13.932669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:13.944528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 973/1929] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/projects => generated 385 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:13.949762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.957772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 957/1930] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:13.959289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:13.975291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8194e5-04a2-4791-9a5f-96df64b780f9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.115491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6637f-d965-430a-955f-f07263a71e5d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.117852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.120574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.120868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.121122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.121242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.121812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.121919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.122278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.122621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.123388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.126119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.134316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.161811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 974/1931] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:14.163223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.182698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2848e29-bff6-4859-b27b-c1798f6eaaac admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.195502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 958/1932] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:13 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:14.198589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.199276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.199559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.199833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.200044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.200497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.202123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.228841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 975/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:14.233689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.257807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944996b-0575-4a4a-bb64-f5cd65025d9f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.298334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:14.326170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ff0676-eabe-4c50-87d1-c0fe0097066a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.334022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.334655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.334932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.335202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.336014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.338806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.371447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.372450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.372842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.373209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.374038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.375787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.386816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 959/1934] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/users => generated 291 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:14.391273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.392762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 976/1935] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:14.397025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.412074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a9c0-04b8-48bc-9e71-cd5e6ceeed45 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.412552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fcbd0-73bf-4308-9de2-9782e035d06c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.421769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.422528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.422777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfc3df-2c06-4986-85c9-ab2cd12104fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.422966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.423216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.423941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc3df-2c06-4986-85c9-ab2cd12104fa admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.424049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.424277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc3df-2c06-4986-85c9-ab2cd12104fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.424677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfc3df-2c06-4986-85c9-ab2cd12104fa admin admin] PATH_INFO: `/v3/projects/152a03466674426da48fdaaca3f03eb2/users/08eae61f786a4c55bfe9216b00afa028/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.425860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.453704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 960/1936] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:14.459482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfc3df-2c06-4986-85c9-ab2cd12104fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=152a03466674426da48fdaaca3f03eb2, user_id=08eae61f786a4c55bfe9216b00afa028, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.463209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfc3df-2c06-4986-85c9-ab2cd12104fa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.467111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.485119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177be4d5-1581-428d-a536-fafd90b0c1af admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.497067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.500482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 977/1937] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:14 2022] PUT /identity/v3/projects/152a03466674426da48fdaaca3f03eb2/users/08eae61f786a4c55bfe9216b00afa028/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:14.500694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.500782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.501052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.501704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.504209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.514118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.514472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.514679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.531815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 961/1938] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:14.536559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:14.538065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.554700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b90365b-243e-411d-8cda-52f93d4d3614 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.562159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] MFA Rules not processed for user `08eae61f786a4c55bfe9216b00afa028`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:14.564314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.655960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff573c39-8b87-4e75-9167-eeb9e444f616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.657526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573c39-8b87-4e75-9167-eeb9e444f616 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.657815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573c39-8b87-4e75-9167-eeb9e444f616 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.658063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff573c39-8b87-4e75-9167-eeb9e444f616 admin admin] PATH_INFO: `/v3/projects/e9f2994e5ff148d79f86b7fb26c42009/users/f8fea3da1027488fa8718a6865299597/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.680960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573c39-8b87-4e75-9167-eeb9e444f616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f2994e5ff148d79f86b7fb26c42009, user_id=f8fea3da1027488fa8718a6865299597, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:14.683989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff573c39-8b87-4e75-9167-eeb9e444f616 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:14.702777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 978/1939] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:14.706896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 962/1940] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:14 2022] PUT /identity/v3/projects/e9f2994e5ff148d79f86b7fb26c42009/users/f8fea3da1027488fa8718a6865299597/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:14.716708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.718286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.722612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.723138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.723611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.755667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] MFA Rules not processed for user `f8fea3da1027488fa8718a6865299597`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:14.756955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:14.806109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b158f-fecc-4de1-b1cc-0777a9cf97d2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:14.882286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 963/1941] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:14.887336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:14.894686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:14.896293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:14.896662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:14.896998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:14.980090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.006913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.010332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.022895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 979/1942] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:15.034321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7bb17-4e04-451c-91c1-fa1fdae8d278 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.046213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.051426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.052063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.052386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.052656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.054541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.055397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.076530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e2f811-6139-4d38-b448-2724a2d775d9 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.091350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.091963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.092430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.092597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] PATH_INFO: `/v3/users/08eae61f786a4c55bfe9216b00afa028` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.094615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08eae61f786a4c55bfe9216b00afa028)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.099048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.102118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.120762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 964/1943] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:14 2022] POST /identity/v3/projects => generated 381 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:15.121874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.203375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08eae61f786a4c55bfe9216b00afa028'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.223893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08eae61f786a4c55bfe9216b00afa028'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.224620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08eae61f786a4c55bfe9216b00afa028'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.225706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08eae61f786a4c55bfe9216b00afa028'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.239789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc0e6c73-6d52-4b93-ac73-be735c7954f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08eae61f786a4c55bfe9216b00afa028'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:15.240043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdae313-21df-4810-9e43-6b5fae32deac admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.247057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.247829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.248112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.248572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.249218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.251505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.284951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 980/1944] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:15 2022] DELETE /identity/v3/users/08eae61f786a4c55bfe9216b00afa028 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:15.286550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.289658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 965/1945] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:15 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:15.289821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.289821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.291169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.293314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.307554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae19c5b-502b-48df-ba8f-2b84e571eb4c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.312236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:15.313787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.387074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.387848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.388394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.388811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.389839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.392876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.409601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 981/1946] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:15.416613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 966/1947] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:15.417982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.423407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.426856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.428522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0ad0b-8e8a-41a7-906d-41d40a0545f1 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.433997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.434857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.434989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.435244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] PATH_INFO: `/v3/projects/a14e96dd6fcf4dee9ac951225981e360/users/4d7cbc72cab64174a618bc90f392cc85/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.454275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14e96dd6fcf4dee9ac951225981e360, user_id=4d7cbc72cab64174a618bc90f392cc85, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.457626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.501780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 982/1948] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:15 2022] PUT /identity/v3/projects/a14e96dd6fcf4dee9ac951225981e360/users/4d7cbc72cab64174a618bc90f392cc85/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:15.503761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.505153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.524789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20e6a2-d87c-4c4f-beaf-cd21ff338948 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.581383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:15.588883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb39535-3dd0-4edc-9177-d99d1a167954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.589775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb39535-3dd0-4edc-9177-d99d1a167954 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.590062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb39535-3dd0-4edc-9177-d99d1a167954 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.592065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb39535-3dd0-4edc-9177-d99d1a167954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.596199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c26822-0934-46d7-8c00-ad7b652be3f1 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.603827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.604604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.605062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.605455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.606198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.608040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.613247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb39535-3dd0-4edc-9177-d99d1a167954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.615682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb39535-3dd0-4edc-9177-d99d1a167954 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.621132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 983/1949] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:15 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:15.625296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.630584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 967/1950] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:15.637662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873af395-6e47-4406-b566-14149fe063a8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:15.642706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:15.643388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.643711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.643989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] PATH_INFO: `/v3/projects/a14e96dd6fcf4dee9ac951225981e360/users/4d7cbc72cab64174a618bc90f392cc85/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.646481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a14e96dd6fcf4dee9ac951225981e360, user_id=4d7cbc72cab64174a618bc90f392cc85, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:15.649153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:15.673152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 984/1951] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:15 2022] PUT /identity/v3/projects/a14e96dd6fcf4dee9ac951225981e360/users/4d7cbc72cab64174a618bc90f392cc85/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:15.676067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:15.676641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:15.677146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:15.720739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] MFA Rules not processed for user `4d7cbc72cab64174a618bc90f392cc85`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:15.723646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:15.938407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 968/1952] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:15.950826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:15.953332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.126626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90593752-bf24-4c58-8356-f594a634fcdb admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.133086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.133848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.134352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.135515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.152356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.154751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.173898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 985/1953] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:16.498032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.498032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.605862 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c25e83a-790c-4363-bd0d-edaee0683da9 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.610493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.611169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.611396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.611745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] PATH_INFO: `/v3/projects/152a03466674426da48fdaaca3f03eb2` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.612997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=152a03466674426da48fdaaca3f03eb2)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.614935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.647707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:16.658847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.670768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e13008-d1d5-48b5-a128-5868fa4b44eb service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.677040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '152a03466674426da48fdaaca3f03eb2', 'request_id': 'req-789856ad-5513-4f96-8b50-c61503c925d4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.684128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '152a03466674426da48fdaaca3f03eb2', 'request_id': 'req-789856ad-5513-4f96-8b50-c61503c925d4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.694269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '152a03466674426da48fdaaca3f03eb2', 'request_id': 'req-789856ad-5513-4f96-8b50-c61503c925d4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.709837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] The token cache is being invalidate because project 152a03466674426da48fdaaca3f03eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:16.710239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.710903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-789856ad-5513-4f96-8b50-c61503c925d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.712818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 969/1954] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:16 2022] DELETE /identity/v3/projects/152a03466674426da48fdaaca3f03eb2 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:16.718673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59d78-d9c1-427a-9284-9a8277d8bddc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.718982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59d78-d9c1-427a-9284-9a8277d8bddc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.719365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f59d78-d9c1-427a-9284-9a8277d8bddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.748849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-96f59d78-d9c1-427a-9284-9a8277d8bddc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.750466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-96f59d78-d9c1-427a-9284-9a8277d8bddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.771333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.773170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.774070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.774682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] PATH_INFO: `/v3/users/f8fea3da1027488fa8718a6865299597` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.776861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8fea3da1027488fa8718a6865299597)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.780769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.862759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8fea3da1027488fa8718a6865299597'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.864590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 970/1955] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.871390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8fea3da1027488fa8718a6865299597'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.872010 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8fea3da1027488fa8718a6865299597'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.872671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8fea3da1027488fa8718a6865299597'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.873680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.874108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.874387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.880081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-67a67ee3-2609-40e4-a3c5-5771f9f5db5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8fea3da1027488fa8718a6865299597'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:16.896264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.897415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:16.897864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.903012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 986/1956] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:16 2022] DELETE /identity/v3/users/f8fea3da1027488fa8718a6865299597 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:16.907863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.908255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.908535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.911327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 971/1957] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:16.915451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:16.924760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d8854c-2046-4b87-9a32-6a7a4ad2221a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:16.929475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:16.930733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:16.981855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:16.982487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:16.982765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:16.983036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:16.983677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:16.983976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:16.985562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:16.992735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 987/1958] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:17.000829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 972/1959] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:17.002054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.004379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.006049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.015908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a651ce-c014-4d49-ba1c-1e33b35c2aad None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.021915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.022533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.022818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.023087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.023718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.026216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.054642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.061468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 988/1960] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:17.065708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.076731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01944ed-58e8-4779-a06a-29358db44275 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.132122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e38ba-8a48-45ef-bcab-46dd9a417944 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.137154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.137795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.138063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.138429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.139110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.139253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.139829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.140107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.140384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.141033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.150725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.153913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.169616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 989/1961] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:17.169795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 973/1962] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:17.171213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.182544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049983eb-7e41-42bd-9f78-38a24e0a6810 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.190613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.191292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.191646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.192002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.192726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.194744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.206609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 990/1963] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:17.210656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.221306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928bb213-9fb2-4024-a0da-74e3ff044909 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.226886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d1551d-4857-4d0b-af43-09d58c43034a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.227610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1551d-4857-4d0b-af43-09d58c43034a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.228454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1551d-4857-4d0b-af43-09d58c43034a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.228761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d1551d-4857-4d0b-af43-09d58c43034a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.229398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1551d-4857-4d0b-af43-09d58c43034a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.231008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d1551d-4857-4d0b-af43-09d58c43034a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.245583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 974/1964] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:17.249706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.262038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adc0704-e2b8-474c-b1be-e9aaa4962bc2 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.273718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.274779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.275162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.275565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] PATH_INFO: `/v3/projects/b4123a34c39c4293b3d7c9efb873c9c4/users/9436c79c993c4ecbb2346362960d0fd7/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.319908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4123a34c39c4293b3d7c9efb873c9c4, user_id=9436c79c993c4ecbb2346362960d0fd7, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.322757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.349664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 991/1965] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:17 2022] PUT /identity/v3/projects/b4123a34c39c4293b3d7c9efb873c9c4/users/9436c79c993c4ecbb2346362960d0fd7/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:17.361237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.361237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.361493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.430047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] MFA Rules not processed for user `9436c79c993c4ecbb2346362960d0fd7`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:17.432266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.510257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 975/1966] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:17.521563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.523419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.567325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.590970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.592995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.595338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487df7f-2dae-4a92-bace-ba98f65cff3c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.599778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.600414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.600698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.600970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.613406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.614942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.624722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 992/1967] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:17 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:17.650922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.676607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a378c4-e6a5-4333-aa8b-c255e9000ae4 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.681963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.683064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.683411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.683893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] PATH_INFO: `/v3/projects/e9f2994e5ff148d79f86b7fb26c42009` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.685461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f2994e5ff148d79f86b7fb26c42009)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.687761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.714069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:17.740202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a94a44-e7ad-4015-8018-0a397071f931 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.746157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f2994e5ff148d79f86b7fb26c42009', 'request_id': 'req-be4f1f6b-3b72-4d9a-bced-39225fd4770d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.753825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f2994e5ff148d79f86b7fb26c42009', 'request_id': 'req-be4f1f6b-3b72-4d9a-bced-39225fd4770d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.762121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f2994e5ff148d79f86b7fb26c42009', 'request_id': 'req-be4f1f6b-3b72-4d9a-bced-39225fd4770d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.773846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] The token cache is being invalidate because project e9f2994e5ff148d79f86b7fb26c42009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:17.774142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.775334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.777171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 976/1968] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:17 2022] DELETE /identity/v3/projects/e9f2994e5ff148d79f86b7fb26c42009 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:17.781190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:17.783408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:17.808189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.808864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.809148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.809427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] PATH_INFO: `/v3/users/9436c79c993c4ecbb2346362960d0fd7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.811102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9436c79c993c4ecbb2346362960d0fd7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.813887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.891966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9436c79c993c4ecbb2346362960d0fd7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.896512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4f1f6b-3b72-4d9a-bced-39225fd4770d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:17.908668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9436c79c993c4ecbb2346362960d0fd7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.909043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:17.909684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.909822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9436c79c993c4ecbb2346362960d0fd7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.910065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.910264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] PATH_INFO: `/v3/users/4d7cbc72cab64174a618bc90f392cc85` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:17.911697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9436c79c993c4ecbb2346362960d0fd7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.915027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7cbc72cab64174a618bc90f392cc85)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:17.922686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:17.926748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2e232d55-14a1-4f0f-a4f2-b86348dac250 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9436c79c993c4ecbb2346362960d0fd7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:17.978323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 993/1969] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:17 2022] DELETE /identity/v3/users/9436c79c993c4ecbb2346362960d0fd7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:17.979815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:17.980101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:17.980374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.006089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7cbc72cab64174a618bc90f392cc85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.013239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.033114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.036491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.037365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7cbc72cab64174a618bc90f392cc85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.038134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7cbc72cab64174a618bc90f392cc85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.038791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7cbc72cab64174a618bc90f392cc85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.062007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7cbc72cab64174a618bc90f392cc85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.123732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 977/1970] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:17 2022] DELETE /identity/v3/users/4d7cbc72cab64174a618bc90f392cc85 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:18.189537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 994/1971] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.201152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.203538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.245566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.279879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84523c03-9650-4452-bbad-9c8adc14b922 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.283716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5216a9c-7e6e-483d-810a-bd06088ef9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.284315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216a9c-7e6e-483d-810a-bd06088ef9df service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.284587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216a9c-7e6e-483d-810a-bd06088ef9df service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.284854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5216a9c-7e6e-483d-810a-bd06088ef9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.364704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5216a9c-7e6e-483d-810a-bd06088ef9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.366179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5216a9c-7e6e-483d-810a-bd06088ef9df service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.376225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 978/1972] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:18.442828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.444310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.457428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4d88f-1c38-4bf1-93e8-f9b59b6e369f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.463125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.463765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.464040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.464301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] PATH_INFO: `/v3/projects/a14e96dd6fcf4dee9ac951225981e360` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.465480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a14e96dd6fcf4dee9ac951225981e360)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.467416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.544145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a14e96dd6fcf4dee9ac951225981e360', 'request_id': 'req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.551431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a14e96dd6fcf4dee9ac951225981e360', 'request_id': 'req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.566139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a14e96dd6fcf4dee9ac951225981e360', 'request_id': 'req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.581547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] The token cache is being invalidate because project a14e96dd6fcf4dee9ac951225981e360 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:18.581968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.583181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.585772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 995/1973] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:18 2022] DELETE /identity/v3/projects/a14e96dd6fcf4dee9ac951225981e360 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:18.592221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df443-51bf-4202-9e49-66a588e258bc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.592702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df443-51bf-4202-9e49-66a588e258bc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.592781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6df443-51bf-4202-9e49-66a588e258bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.623204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3c6df443-51bf-4202-9e49-66a588e258bc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.624912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3c6df443-51bf-4202-9e49-66a588e258bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.729437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 979/1974] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.730980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.733854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.738919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.739206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.739485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.749097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd013cf1-6684-4d05-b80f-5eb5b8c8f0b0 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.756157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.757129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.757493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.757874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] PATH_INFO: `/v3/projects/b4123a34c39c4293b3d7c9efb873c9c4` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.759949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4123a34c39c4293b3d7c9efb873c9c4)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.762093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.782833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.784105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.798433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 980/1975] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.800312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:18.817801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2f482-9413-4bd6-9893-c0cb2cf39459 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:18.843395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4123a34c39c4293b3d7c9efb873c9c4', 'request_id': 'req-f045a0fe-4f42-4f48-9867-313634887d99'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.851562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4123a34c39c4293b3d7c9efb873c9c4', 'request_id': 'req-f045a0fe-4f42-4f48-9867-313634887d99'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.862494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4123a34c39c4293b3d7c9efb873c9c4', 'request_id': 'req-f045a0fe-4f42-4f48-9867-313634887d99'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.880972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] The token cache is being invalidate because project b4123a34c39c4293b3d7c9efb873c9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:18.881260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.887615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f045a0fe-4f42-4f48-9867-313634887d99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:18.890264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 996/1976] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:18 2022] DELETE /identity/v3/projects/b4123a34c39c4293b3d7c9efb873c9c4 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:18.900302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a1ae-d812-4278-98ee-1b25d029bb7e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.900894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a1ae-d812-4278-98ee-1b25d029bb7e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.902192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a1ae-d812-4278-98ee-1b25d029bb7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.944961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-13d5a1ae-d812-4278-98ee-1b25d029bb7e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:18.946364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-13d5a1ae-d812-4278-98ee-1b25d029bb7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:18.947089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:18.948141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.948429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.949148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.950113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:18.950470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:18.953808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:18.959251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 997/1977] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:18.969588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:18.970147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:18.970428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:18.985480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:18.988330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 981/1978] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:18.995010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.006270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.018499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.020352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.022518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.040047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0448ec-e99b-4a7e-960f-91c661106f2c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.049461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.050444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.051022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.051362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.054978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 998/1979] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:19.055349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.055349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.056848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.073414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7b9ea-66ee-4a3a-8c10-fd2371b2bc06 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.105258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.106688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.108517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.108517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.108517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:19.108517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.110123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.111337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 982/1980] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:18 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:19.122081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.127434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 999/1981] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 19:51:19.128922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.170794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504af66-9c4e-4139-8eca-6bae88b64e63 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.247139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d73383-7e43-4f7a-b853-259ee69e1c76 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.253623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.254099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.254727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.255080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.255788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.258100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.287011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 983/1982] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/users => generated 293 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:19.291461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.296386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.297214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.297636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.298093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.298879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.301438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.305578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df239859-f0e4-4be7-8138-3df7f7a765c1 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.311292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.312113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.312493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.312849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.314113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.315483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.330333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 984/1983] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:19.334299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.345957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1000/1984] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/projects => generated 375 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:19.349254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6ad67c-ff29-43d4-899e-ed0c36d57a3d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.351529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.437821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.438526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.438879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.439275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.440202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.443135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.444520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d1b0b-d8d4-49d8-91f4-2d3a94c35455 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.450210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.450769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.451050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.451323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.451963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.454581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.456832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 985/1985] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:19.461350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.473473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7435350a-dbc8-4a65-b647-ae33bf48c971 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.479879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823e9966-118e-4fc6-8a5f-988413d64429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.480811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9966-118e-4fc6-8a5f-988413d64429 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.481316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9966-118e-4fc6-8a5f-988413d64429 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.481801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823e9966-118e-4fc6-8a5f-988413d64429 admin admin] PATH_INFO: `/v3/projects/0a206baa5dde422baf73f756985f3556/users/c801fd449cc840e3872f677cb51e7a4e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.482432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1001/1986] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/users => generated 288 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:19.486318 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.497124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b07d640-4c93-460b-8fd0-582d0e6aade6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.501996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e9966-118e-4fc6-8a5f-988413d64429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a206baa5dde422baf73f756985f3556, user_id=c801fd449cc840e3872f677cb51e7a4e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.502776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.503385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.503686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.503961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.504605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.506138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823e9966-118e-4fc6-8a5f-988413d64429 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.506347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.526185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1002/1987] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:19.534020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.535785 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 986/1988] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:19 2022] PUT /identity/v3/projects/0a206baa5dde422baf73f756985f3556/users/c801fd449cc840e3872f677cb51e7a4e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:19.548358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906c1fe1-f92c-495d-a9b3-ecf231d028ee admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.552554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.552863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.553166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.586815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] MFA Rules not processed for user `c801fd449cc840e3872f677cb51e7a4e`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.589093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.605071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.611552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:19.621978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.622591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.622957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.623241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.623883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.625484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.637767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1003/1989] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:19.640636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.655240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e48756-ede5-4708-81b5-992c0e7834cf admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.662049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 987/1990] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:19.666732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaadbbb-78e4-4e62-8b4a-f49c04c43562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.667410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaadbbb-78e4-4e62-8b4a-f49c04c43562 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.667691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaadbbb-78e4-4e62-8b4a-f49c04c43562 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.667965 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaadbbb-78e4-4e62-8b4a-f49c04c43562 admin admin] PATH_INFO: `/v3/projects/7a648994ee6d4c239ce6591396a455c5/users/b43021150baf4f9c807fb6a29049070b/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.674866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.676325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.683943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaadbbb-78e4-4e62-8b4a-f49c04c43562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a648994ee6d4c239ce6591396a455c5, user_id=b43021150baf4f9c807fb6a29049070b, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.686636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaadbbb-78e4-4e62-8b4a-f49c04c43562 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.701152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1004/1991] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:19 2022] PUT /identity/v3/projects/7a648994ee6d4c239ce6591396a455c5/users/b43021150baf4f9c807fb6a29049070b/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:19.711051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.711511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.711783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.737037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9197712-d98b-4fbf-ac1a-cff8af3fc67f None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.740094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] MFA Rules not processed for user `b43021150baf4f9c807fb6a29049070b`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:19.741325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.786528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.787256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.787532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.787818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.806347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1005/1992] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:19.817102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:19.818454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:19.839448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc63e7bf-1abf-46ec-9e0c-e36c8b958f0c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:19.843279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642dea9-f806-4d9f-bc1e-0c1453d5a8ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:19.843889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642dea9-f806-4d9f-bc1e-0c1453d5a8ae service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:19.844168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642dea9-f806-4d9f-bc1e-0c1453d5a8ae service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:19.844440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642dea9-f806-4d9f-bc1e-0c1453d5a8ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:19.846175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.847590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.853092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642dea9-f806-4d9f-bc1e-0c1453d5a8ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:19.854547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e642dea9-f806-4d9f-bc1e-0c1453d5a8ae service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:19.856450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 988/1993] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:19.863343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1006/1994] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:29.852379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=85 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:29.859316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=56,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:31.466536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.578440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6186ee6f-2e0a-45db-8c4f-5b72142367eb service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.583444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.584226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.584493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.584767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] PATH_INFO: `/v3/users/b43021150baf4f9c807fb6a29049070b` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.586220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b43021150baf4f9c807fb6a29049070b)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.588368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.645229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b43021150baf4f9c807fb6a29049070b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.653372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b43021150baf4f9c807fb6a29049070b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.654092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b43021150baf4f9c807fb6a29049070b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.654900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b43021150baf4f9c807fb6a29049070b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.663293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b43021150baf4f9c807fb6a29049070b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:31.686057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 989/1995] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:31 2022] DELETE /identity/v3/users/b43021150baf4f9c807fb6a29049070b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:31.690484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a97b6e-bc7f-4bcc-a3bc-36b670bb4bb8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.691340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a97b6e-bc7f-4bcc-a3bc-36b670bb4bb8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.691619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a97b6e-bc7f-4bcc-a3bc-36b670bb4bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.712003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d3a97b6e-bc7f-4bcc-a3bc-36b670bb4bb8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:31.713239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d3a97b6e-bc7f-4bcc-a3bc-36b670bb4bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.778760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1007/1996] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:31.789026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.790492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.856923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980942-3fdb-40f9-93a0-39b031142ac0 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.860617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.861160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.861440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.861713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.870758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.872743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.882657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 990/1997] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:31.937176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148ff07-215d-44e4-957b-513f6c2a12dc None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.937544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148ff07-215d-44e4-957b-513f6c2a12dc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.937940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1148ff07-215d-44e4-957b-513f6c2a12dc None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.939586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1008/1998] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:31.948779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:31.950234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:31.965935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f570161d-d827-4096-a2f0-a0bd06b5ea2c service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:31.971371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:31.972133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:31.972539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:31.972932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] PATH_INFO: `/v3/users/c801fd449cc840e3872f677cb51e7a4e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:31.974257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c801fd449cc840e3872f677cb51e7a4e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:31.976316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:31.998952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:32.079398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c801fd449cc840e3872f677cb51e7a4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.099966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c801fd449cc840e3872f677cb51e7a4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.102606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c801fd449cc840e3872f677cb51e7a4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.103709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c801fd449cc840e3872f677cb51e7a4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.115125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a05b326f-71b6-477e-9044-06d0db00dd1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c801fd449cc840e3872f677cb51e7a4e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.139739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 991/1999] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:31 2022] DELETE /identity/v3/users/c801fd449cc840e3872f677cb51e7a4e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:32.143792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.144077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.144353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.165492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.166772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.190558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.192190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.192921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.233069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.235467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.248425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1009/2000] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.251526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.251526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.264539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841e68a-e50c-4865-96e1-e3c88335e790 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.274144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.276242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.277189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.277485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] PATH_INFO: `/v3/projects/7a648994ee6d4c239ce6591396a455c5` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.278972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a648994ee6d4c239ce6591396a455c5)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.281155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.381212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a648994ee6d4c239ce6591396a455c5', 'request_id': 'req-72724115-e3f6-41c8-bab6-c75c35b51734'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.390323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a648994ee6d4c239ce6591396a455c5', 'request_id': 'req-72724115-e3f6-41c8-bab6-c75c35b51734'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.397841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 992/2001] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.398934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.399849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a648994ee6d4c239ce6591396a455c5', 'request_id': 'req-72724115-e3f6-41c8-bab6-c75c35b51734'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.400699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.427962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] The token cache is being invalidate because project 7a648994ee6d4c239ce6591396a455c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:32.428275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.429316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-72724115-e3f6-41c8-bab6-c75c35b51734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:32.431250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1010/2002] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:32 2022] DELETE /identity/v3/projects/7a648994ee6d4c239ce6591396a455c5 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:32.432953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32254b-d8e4-4f7b-b288-bf9d9f7a867a None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.433236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32254b-d8e4-4f7b-b288-bf9d9f7a867a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.433516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd32254b-d8e4-4f7b-b288-bf9d9f7a867a None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.437890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1011/2003] 10.222.0.15 () {54 vars in 961 bytes} [Tue Sep 13 19:51:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 19:51:32.439762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f6611-64c2-43cc-a93f-90aa6edf8ab7 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.440123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f6611-64c2-43cc-a93f-90aa6edf8ab7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.440653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f6611-64c2-43cc-a93f-90aa6edf8ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.466435 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-663f6611-64c2-43cc-a93f-90aa6edf8ab7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.467913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-663f6611-64c2-43cc-a93f-90aa6edf8ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.491302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6c3b6-b6d8-4efc-8597-4cab7de4c0e0 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.508295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:32.564414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:32.572728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.574281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.577019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.583291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.587787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.616434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.619304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876b77e9-6a1f-4f89-b5ad-a7623415b541 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.624112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1012/2004] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.641522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.641522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.641522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 993/2005] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:32.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.642083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.675410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.676651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:32.677373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.678346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.687162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 994/2006] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.691434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.710193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb066cb6-a7ff-4a7a-8aa3-d1b48db8bec9 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.717067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.717972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.719274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.719777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.721018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:32.721980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.724093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.755443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 995/2007] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:32.761820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.774533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6241c58-36b1-445b-be56-ba8398efc25b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.780375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.781058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.781341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.781611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.782565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.784056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1013/2008] 10.222.0.15 () {58 vars in 1058 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:32.784580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.785045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.803750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f25c4ca-98d1-47c9-b037-5a742f90803c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.812951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.813622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.813933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.814781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.816063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:32.821591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 996/2009] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/projects => generated 369 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:32.822510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.833140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.835769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.855152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1014/2010] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:32.908348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a34f63a-b422-40c5-8560-6d17be7cf819 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.913449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.914183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.914584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.914902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.915546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.919008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:32.976742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 997/2011] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:32 2022] POST /identity/v3/users => generated 285 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:32.977962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:32.990891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07cc4e4-67d5-4998-a57e-2a1ab2a49fed service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:32.995797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:32.996466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:32.996786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:32.997059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:32.997770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:32.999314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.015973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1015/2012] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:33.017601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.032750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8d52d3-c49e-4c28-95da-af50d10151bf admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.044078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f47043-5e97-4e58-9add-e53dff4e7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.044705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47043-5e97-4e58-9add-e53dff4e7aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.044981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47043-5e97-4e58-9add-e53dff4e7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.045367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f47043-5e97-4e58-9add-e53dff4e7aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.047608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f47043-5e97-4e58-9add-e53dff4e7aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.051637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f47043-5e97-4e58-9add-e53dff4e7aa7 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.074597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 998/2013] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:33.077199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.092795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0739e74-d5f5-4855-973b-0572f1b2027a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.104677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.105490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.106139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.106510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] PATH_INFO: `/v3/projects/d31852d50f034c9a85bd20a8245bad38/users/214e960097d248ba97e3d852d25cb075/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.150006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31852d50f034c9a85bd20a8245bad38, user_id=214e960097d248ba97e3d852d25cb075, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.153177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.181800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1016/2014] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:33 2022] PUT /identity/v3/projects/d31852d50f034c9a85bd20a8245bad38/users/214e960097d248ba97e3d852d25cb075/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:33.185886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.186437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.186779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.212001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.214125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.253654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] MFA Rules not processed for user `214e960097d248ba97e3d852d25cb075`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:33.259463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.371056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74c457-f4d1-47fe-9637-e65a5a81d9ca admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.398053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.398790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.399170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.399527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] PATH_INFO: `/v3/users/50cd0f5e6a2e426db9901e6f47693338` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.400915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50cd0f5e6a2e426db9901e6f47693338)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.405277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.418137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 999/2015] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:33.419820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.422970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.430355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.441934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcf06d-e4f0-4889-a9df-d744d4a9eeff None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.449373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.450158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.450548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.450885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] PATH_INFO: `/v3/projects/0a206baa5dde422baf73f756985f3556` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.452164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a206baa5dde422baf73f756985f3556)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.454219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.467409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.482548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50cd0f5e6a2e426db9901e6f47693338'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.490723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50cd0f5e6a2e426db9901e6f47693338'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.491497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50cd0f5e6a2e426db9901e6f47693338'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.492204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50cd0f5e6a2e426db9901e6f47693338'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.501546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50cd0f5e6a2e426db9901e6f47693338'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.516426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a206baa5dde422baf73f756985f3556', 'request_id': 'req-84efb518-7022-4f99-a452-317d2518ffec'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.523905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1017/2016] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:33 2022] DELETE /identity/v3/users/50cd0f5e6a2e426db9901e6f47693338 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:33.524639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.526816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.531675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a206baa5dde422baf73f756985f3556', 'request_id': 'req-84efb518-7022-4f99-a452-317d2518ffec'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.545771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a206baa5dde422baf73f756985f3556', 'request_id': 'req-84efb518-7022-4f99-a452-317d2518ffec'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.563366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] The token cache is being invalidate because project 0a206baa5dde422baf73f756985f3556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:33.563754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.564515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84efb518-7022-4f99-a452-317d2518ffec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.566445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1000/2017] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:33 2022] DELETE /identity/v3/projects/0a206baa5dde422baf73f756985f3556 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:33.572586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1ef0a-4895-4fd5-9369-abfe1026f6c4 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.572998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1ef0a-4895-4fd5-9369-abfe1026f6c4 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.573344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c1ef0a-4895-4fd5-9369-abfe1026f6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.603955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b7c1ef0a-4895-4fd5-9369-abfe1026f6c4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:33.605423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b7c1ef0a-4895-4fd5-9369-abfe1026f6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.653533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c1eb49-0b53-42b3-873e-0ca254f309c5 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.662087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.663278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.663996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.664590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.667394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.722623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1001/2018] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:33.731344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.732452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.733321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.758995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:33.760604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.773054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1002/2019] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:33.777667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.787271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.788654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.795632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58b527-a8ad-46c5-92d5-81e4e4c51696 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.798162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1018/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:33.815910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.816663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.816948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.817218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.818387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:33.818690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.820478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.837761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1003/2021] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:33.839380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.856529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cf2e26-e349-49f3-a093-e06c8de513ad service glance] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.860967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.863192 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:33.868525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.869165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.869165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.869432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.870155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.872579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.890613 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63cebb-2b86-495d-988d-8bd6401ac503 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:33.898338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:33.899118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:33.899512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:33.899853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] PATH_INFO: `/v3/projects/dba514425319450f9606867f4bd48664` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:33.901759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dba514425319450f9606867f4bd48664)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:33.911703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:33.921876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1019/2022] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:33.927844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:33.934938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:33.987337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dba514425319450f9606867f4bd48664', 'request_id': 'req-a69f4bd9-7c26-454c-9ce7-70f323e63568'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:33.997723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dba514425319450f9606867f4bd48664', 'request_id': 'req-a69f4bd9-7c26-454c-9ce7-70f323e63568'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.006006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dba514425319450f9606867f4bd48664', 'request_id': 'req-a69f4bd9-7c26-454c-9ce7-70f323e63568'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.010496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b32319-b06b-4c93-9660-c35903db7f2e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.020700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] The token cache is being invalidate because project dba514425319450f9606867f4bd48664 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:34.021045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.021924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.023540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1004/2023] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:33 2022] DELETE /identity/v3/projects/dba514425319450f9606867f4bd48664 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:34.027277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.028806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.035183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:34.093567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.094569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.094912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.095250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.095959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.097883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.114682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69f4bd9-7c26-454c-9ce7-70f323e63568 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.119576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.120046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1020/2024] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:33 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:34.120277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.120555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.120913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] PATH_INFO: `/v3/users/707a84a0ad384c7bb40deccfbe07b4bb` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.122593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=707a84a0ad384c7bb40deccfbe07b4bb)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.124777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.124893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.128357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.147570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d632-de92-4287-bb59-65fe82fcfe7f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.156389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.157916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.158382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.158822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.159841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.161685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.175669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1021/2025] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:34.179884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.194185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3b90e-ec9e-4c17-80ed-70b12ad45129 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.198452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '707a84a0ad384c7bb40deccfbe07b4bb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.200877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.201495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.201806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.202084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.203088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.205295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.210703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '707a84a0ad384c7bb40deccfbe07b4bb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.212377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '707a84a0ad384c7bb40deccfbe07b4bb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.213347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '707a84a0ad384c7bb40deccfbe07b4bb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.218740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1022/2026] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:34 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:34.223084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f7169a2d-72f9-4eeb-9de4-13ae1601fdfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '707a84a0ad384c7bb40deccfbe07b4bb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.223258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.226270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.242189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98225-716d-40ff-ab26-1c0e8809fa21 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.249851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1005/2027] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:34 2022] DELETE /identity/v3/users/707a84a0ad384c7bb40deccfbe07b4bb => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:34.252982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.253666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.253985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.254256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] PATH_INFO: `/v3/projects/d55772ef51484970b1f8dab96ff480a3/users/b9ff584fb6e44ad1b5ec59710813e100/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.272486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d55772ef51484970b1f8dab96ff480a3, user_id=b9ff584fb6e44ad1b5ec59710813e100, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.276649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.298302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1023/2028] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:34 2022] PUT /identity/v3/projects/d55772ef51484970b1f8dab96ff480a3/users/b9ff584fb6e44ad1b5ec59710813e100/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:34.310065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.310353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.310654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.346109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] MFA Rules not processed for user `b9ff584fb6e44ad1b5ec59710813e100`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:34.348103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.466150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1006/2029] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:34.505997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.507770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.633192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b32485-f317-4e7c-a89e-a1d7dcede4b9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.641873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.642513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.642819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.643063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.652739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.654746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.666646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.669913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.673394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:34.684931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:34.687641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1024/2030] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:34.718442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:34.753863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9325ef97-77af-44c1-b4b8-d84ad66c9ce5 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.761317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.761992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.762306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.762583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] PATH_INFO: `/v3/projects/dcca8cfbbef84a618925a96ef5d75478` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.763967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcca8cfbbef84a618925a96ef5d75478)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.765935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:34.817766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d2f176-f8f3-4f90-96df-a5eae1cd5ba8 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:34.841483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcca8cfbbef84a618925a96ef5d75478', 'request_id': 'req-5e00fcf3-06f1-460d-bf27-840a3651c723'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.848641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcca8cfbbef84a618925a96ef5d75478', 'request_id': 'req-5e00fcf3-06f1-460d-bf27-840a3651c723'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.862503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcca8cfbbef84a618925a96ef5d75478', 'request_id': 'req-5e00fcf3-06f1-460d-bf27-840a3651c723'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.880137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] The token cache is being invalidate because project dcca8cfbbef84a618925a96ef5d75478 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:34.880550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.881305 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5e00fcf3-06f1-460d-bf27-840a3651c723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:34.883143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1007/2031] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:34 2022] DELETE /identity/v3/projects/dcca8cfbbef84a618925a96ef5d75478 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:34.888757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2e38-9675-4186-86ef-98b3f936d134 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.889035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2e38-9675-4186-86ef-98b3f936d134 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.889307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85d2e38-9675-4186-86ef-98b3f936d134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.920439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c85d2e38-9675-4186-86ef-98b3f936d134 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:34.924273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c85d2e38-9675-4186-86ef-98b3f936d134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:34.956409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:34.957242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:34.957835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:34.957948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] PATH_INFO: `/v3/users/b9ff584fb6e44ad1b5ec59710813e100` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:34.962208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ff584fb6e44ad1b5ec59710813e100)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:34.966867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.036606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1008/2032] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:35.046194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.046991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.047559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.048552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ff584fb6e44ad1b5ec59710813e100'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.057306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ff584fb6e44ad1b5ec59710813e100'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.058014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ff584fb6e44ad1b5ec59710813e100'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.058649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ff584fb6e44ad1b5ec59710813e100'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.067277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d49d524e-5a8c-45c8-9c20-1d65598b21bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ff584fb6e44ad1b5ec59710813e100'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.070349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.071710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.080870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1009/2033] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:35.085135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.086836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1025/2034] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:34 2022] DELETE /identity/v3/users/b9ff584fb6e44ad1b5ec59710813e100 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:35.092014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.092345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.092935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.098625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ccdcc8-3db1-4634-97ca-4b4cdaf8fabf None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.137850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.139124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.153684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.170584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.192309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.194373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.194518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.194718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.195536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:35.196367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.197637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.211418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1010/2035] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:35.216060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.229061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b196e6-68d2-43ff-86c8-8793c6574af6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.232398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1026/2036] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:35.234797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.235408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.235689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.236522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.236736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.239911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.242485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.243871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.277070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1011/2037] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/projects => generated 361 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:51:35.278575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.319245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4526b-533e-467c-b387-1fffa59fab33 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.365221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60bf5fd-3c73-46c0-8e71-4ac1766e4383 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.370823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.372217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.372621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.372933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.373786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.376041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.396783 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.397563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.397974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.398378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.406487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1012/2038] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/users => generated 281 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:35.410652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.411128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.412945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.420875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7827bdb-8af8-4093-85ce-4d5087078714 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.424398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1027/2039] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:35.425821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.426457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.427093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.427371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.428084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.429690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.444254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1013/2040] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:35.449381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.462598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bd0c0d-2d4f-4390-af61-2d9166351ff1 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.469447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b2150a-6602-4bb0-a725-6e15d2b6d104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.470182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2150a-6602-4bb0-a725-6e15d2b6d104 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.470533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2150a-6602-4bb0-a725-6e15d2b6d104 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.470873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b2150a-6602-4bb0-a725-6e15d2b6d104 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.471591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2150a-6602-4bb0-a725-6e15d2b6d104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.473405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b2150a-6602-4bb0-a725-6e15d2b6d104 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.489232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1028/2041] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:35.490873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.506259 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15f5a7-9f8d-4915-a533-c388d329ad4d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.512278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.513556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.514043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.521158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] PATH_INFO: `/v3/projects/5641c6a70cd247199dae1cc68057e2c1/users/37f362538e0349419b1e2715b6fe1a66/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.553533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5641c6a70cd247199dae1cc68057e2c1, user_id=37f362538e0349419b1e2715b6fe1a66, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.558143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.588734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1014/2042] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:35 2022] PUT /identity/v3/projects/5641c6a70cd247199dae1cc68057e2c1/users/37f362538e0349419b1e2715b6fe1a66/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:35.600563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.600918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.601176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.660421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] MFA Rules not processed for user `37f362538e0349419b1e2715b6fe1a66`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:35.665341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.739315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1029/2043] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:35.754735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.757364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.777678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.779950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:35.785386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.828975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e45f19-a88d-4a7b-aea2-e54accf98a13 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.833551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.834225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.834499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.834770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] PATH_INFO: `/v3/projects/d55772ef51484970b1f8dab96ff480a3` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.835894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=d55772ef51484970b1f8dab96ff480a3)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.837815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.841275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:35.851500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708bad2f-0026-4331-b92a-b8b9e1373117 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.855232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22414008-1762-4051-88f9-592917c116ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.855840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22414008-1762-4051-88f9-592917c116ec service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.856169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22414008-1762-4051-88f9-592917c116ec service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.856450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22414008-1762-4051-88f9-592917c116ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.869584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22414008-1762-4051-88f9-592917c116ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:35.871296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22414008-1762-4051-88f9-592917c116ec service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:35.881440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1030/2044] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:35.897580 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22414008-1762-4051-88f9-592917c116ec service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:35.897762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd55772ef51484970b1f8dab96ff480a3', 'request_id': 'req-58278535-3990-4926-b86f-5d21a81047ba'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.904700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd55772ef51484970b1f8dab96ff480a3', 'request_id': 'req-58278535-3990-4926-b86f-5d21a81047ba'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.909012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22414008-1762-4051-88f9-592917c116ec service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:35.913022 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd55772ef51484970b1f8dab96ff480a3', 'request_id': 'req-58278535-3990-4926-b86f-5d21a81047ba'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.924962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] The token cache is being invalidate because project d55772ef51484970b1f8dab96ff480a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:35.925342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.926800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-58278535-3990-4926-b86f-5d21a81047ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:35.929023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1015/2045] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:35 2022] DELETE /identity/v3/projects/d55772ef51484970b1f8dab96ff480a3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:35.994172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:35.995046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:35.995393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:35.995742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] PATH_INFO: `/v3/users/37f362538e0349419b1e2715b6fe1a66` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:35.997526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37f362538e0349419b1e2715b6fe1a66)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.000098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.095905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37f362538e0349419b1e2715b6fe1a66'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.109439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37f362538e0349419b1e2715b6fe1a66'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.110373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37f362538e0349419b1e2715b6fe1a66'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.111682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37f362538e0349419b1e2715b6fe1a66'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.122400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37f362538e0349419b1e2715b6fe1a66'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.167767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1031/2046] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:35 2022] DELETE /identity/v3/users/37f362538e0349419b1e2715b6fe1a66 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:51:36.171953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.172240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.172558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.217557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:36.227992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.320738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1016/2047] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:36.332144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.333704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.386082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:36.430614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e7822-aa98-46a7-8eca-b2c1ffba09e4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.436336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f45578-364b-4915-9dd0-6b4bf795fc85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.437373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45578-364b-4915-9dd0-6b4bf795fc85 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.437771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45578-364b-4915-9dd0-6b4bf795fc85 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.438175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f45578-364b-4915-9dd0-6b4bf795fc85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.453361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45578-364b-4915-9dd0-6b4bf795fc85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.455081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f45578-364b-4915-9dd0-6b4bf795fc85 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.468445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1032/2048] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:36.875734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:36.877234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:36.893627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d9f63-1b4c-4f7f-b752-1705837d3fef None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:36.898739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:36.899607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.899951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.900345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] PATH_INFO: `/v3/projects/5641c6a70cd247199dae1cc68057e2c1` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:36.902612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5641c6a70cd247199dae1cc68057e2c1)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:36.905056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:36.953878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5641c6a70cd247199dae1cc68057e2c1', 'request_id': 'req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.960157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5641c6a70cd247199dae1cc68057e2c1', 'request_id': 'req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.968275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5641c6a70cd247199dae1cc68057e2c1', 'request_id': 'req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.978939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] The token cache is being invalidate because project 5641c6a70cd247199dae1cc68057e2c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:36.979225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.979872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-81ebb0a5-844e-4fcf-b32b-32a9ded061eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:36.981400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1017/2049] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:36 2022] DELETE /identity/v3/projects/5641c6a70cd247199dae1cc68057e2c1 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:36.986150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:36.986673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:36.986673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.006838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.008167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.076751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1033/2050] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.085502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.085810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.086087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.105906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.107107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.115138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1018/2051] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.119307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.129497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a6a459-69d3-4709-a42d-a176a8f49da3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.134416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.135014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.135289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.135557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.136227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:37.136529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.139471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.154521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1034/2052] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:37.159194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.168715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3897c27a-1cb4-4782-a18b-e5bdc31ad6a7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.173336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.173958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.174233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.174502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.175136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.177121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.182543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.208367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1019/2053] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.212754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.223089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638d19ee-cc21-4c52-845e-c00320c1f933 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.289701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.290433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.290707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.290977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.291631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.294084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.315919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1035/2054] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:37.320118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.330190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5d629-9df2-45cd-b0b9-4b0800de5f67 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.334650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.335244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.335518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.335786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.336418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.338037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.347881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1020/2055] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.351959 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.362517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39c380-67ce-4651-a257-cfad1d0367ff admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.367081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd5ebc9-1f16-4c86-8180-167640108413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.367707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5ebc9-1f16-4c86-8180-167640108413 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.367987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5ebc9-1f16-4c86-8180-167640108413 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.368258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd5ebc9-1f16-4c86-8180-167640108413 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.368897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5ebc9-1f16-4c86-8180-167640108413 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.370513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd5ebc9-1f16-4c86-8180-167640108413 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.385828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1036/2056] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:37.391897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.404712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6ec9b-40c6-4ed7-8445-e60bedb49d60 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.410806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.411624 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.411910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.412191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] PATH_INFO: `/v3/projects/fb453c35c90c4c578f97a6fa48b8f710/users/ac0e2925e78c4c35b116c8d0e03f576e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.430360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb453c35c90c4c578f97a6fa48b8f710, user_id=ac0e2925e78c4c35b116c8d0e03f576e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.433266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.447598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1021/2057] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:37 2022] PUT /identity/v3/projects/fb453c35c90c4c578f97a6fa48b8f710/users/ac0e2925e78c4c35b116c8d0e03f576e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:37.453132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0ed0c-6843-4739-ab82-656e8a0dd08e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.453496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0ed0c-6843-4739-ab82-656e8a0dd08e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.453892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0ed0c-6843-4739-ab82-656e8a0dd08e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.473256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7ce0ed0c-6843-4739-ab82-656e8a0dd08e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:37.474596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7ce0ed0c-6843-4739-ab82-656e8a0dd08e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.506239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:37.539061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1037/2058] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:37.550391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:37.552155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:37.611135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aad0633-3ab9-45d5-a81a-8772848af3af admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:37.614897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:37.615433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:37.615711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:37.616215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:37.626500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:37.627999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:37.651671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1022/2059] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:38.632108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc81a9-9d01-4ac8-98a7-53820b92e81c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.632739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc81a9-9d01-4ac8-98a7-53820b92e81c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.632955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcc81a9-9d01-4ac8-98a7-53820b92e81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.673487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9bcc81a9-9d01-4ac8-98a7-53820b92e81c None None] MFA Rules not processed for user `ac0e2925e78c4c35b116c8d0e03f576e`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:38.674955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9bcc81a9-9d01-4ac8-98a7-53820b92e81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.781190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1038/2060] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:38.792769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:38.795163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:38.948098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc805cc-cc28-4b48-a528-7ea7f618eed2 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:38.953276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:38.954071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:38.954574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:38.954698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:38.972419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:38.974169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:38.986350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1023/2061] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:51:45.659639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d05dc4-fec4-4c82-aebb-b3853e22c0d2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:45.660209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d05dc4-fec4-4c82-aebb-b3853e22c0d2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:45.660209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d05dc4-fec4-4c82-aebb-b3853e22c0d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:45.684267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-52d05dc4-fec4-4c82-aebb-b3853e22c0d2 None None] MFA Rules not processed for user `ac0e2925e78c4c35b116c8d0e03f576e`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:45.686017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-52d05dc4-fec4-4c82-aebb-b3853e22c0d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:45.693095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1039/2062] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:48.291648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.293143 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.308723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97c84d-2fbd-4dc9-ab9b-c5626c68c1df service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.313683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.314349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.314635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.314905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] PATH_INFO: `/v3/users/0c0d76555c4743e0a37890429260ff61` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.316035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c0d76555c4743e0a37890429260ff61)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.317987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.366871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c0d76555c4743e0a37890429260ff61'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.370395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:48.376166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c0d76555c4743e0a37890429260ff61'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.377100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c0d76555c4743e0a37890429260ff61'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.378217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c0d76555c4743e0a37890429260ff61'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.386340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c0d76555c4743e0a37890429260ff61'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.407709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1024/2063] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:48 2022] DELETE /identity/v3/users/0c0d76555c4743e0a37890429260ff61 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:48.412822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.413450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.413450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.433403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:48.434657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.505790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1040/2064] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:48.518134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.519971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.588151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ba68c-9f5c-4b4f-8c81-aa0abe0c4eac admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.593151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.594306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.594608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.594845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.606238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.607660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.617749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1025/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:51:48.890228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:48.891918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:48.907039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d218c6-6f6c-4eed-be48-1d8763942db6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:48.913630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:48.914743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:48.915486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:48.916400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] PATH_INFO: `/v3/projects/08a1ab177d4140788ef43f270491e970` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:48.919350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a1ab177d4140788ef43f270491e970)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:48.921742 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:48.977968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a1ab177d4140788ef43f270491e970', 'request_id': 'req-91518494-23d3-40ff-a9e1-396dcb9735e9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.980213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:48.985126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a1ab177d4140788ef43f270491e970', 'request_id': 'req-91518494-23d3-40ff-a9e1-396dcb9735e9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:48.994963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a1ab177d4140788ef43f270491e970', 'request_id': 'req-91518494-23d3-40ff-a9e1-396dcb9735e9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.010892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] The token cache is being invalidate because project 08a1ab177d4140788ef43f270491e970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:49.011182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.011840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.013371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1041/2066] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:48 2022] DELETE /identity/v3/projects/08a1ab177d4140788ef43f270491e970 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:49.017395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.026852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72acbd3-2a53-4aec-9b82-c3dac130a7c8 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.086086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.086733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.087007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.087307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] PATH_INFO: `/v3/users/5c10429db20d46b989402c2696f2c29e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.088501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c10429db20d46b989402c2696f2c29e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.091056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.138051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c10429db20d46b989402c2696f2c29e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.145243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c10429db20d46b989402c2696f2c29e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.145926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c10429db20d46b989402c2696f2c29e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.146555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c10429db20d46b989402c2696f2c29e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.154035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-43de58d5-39f3-4c97-87a6-4c36868e8148 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c10429db20d46b989402c2696f2c29e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.161944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:49.173919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1026/2067] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:51:49 2022] DELETE /identity/v3/users/5c10429db20d46b989402c2696f2c29e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:49.455042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.456583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:49.470536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91518494-23d3-40ff-a9e1-396dcb9735e9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.541502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.542229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.542502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.542806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] PATH_INFO: `/v3/projects/1d9a9c8b141844cd8f7c8a08f99b7bf8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.543995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d9a9c8b141844cd8f7c8a08f99b7bf8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.546014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.595098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d9a9c8b141844cd8f7c8a08f99b7bf8', 'request_id': 'req-b9daeead-baed-460e-a8e3-a32dacfe21e0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.601418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d9a9c8b141844cd8f7c8a08f99b7bf8', 'request_id': 'req-b9daeead-baed-460e-a8e3-a32dacfe21e0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.608203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d9a9c8b141844cd8f7c8a08f99b7bf8', 'request_id': 'req-b9daeead-baed-460e-a8e3-a32dacfe21e0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.618386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] The token cache is being invalidate because project 1d9a9c8b141844cd8f7c8a08f99b7bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:51:49.618841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.619587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b9daeead-baed-460e-a8e3-a32dacfe21e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:51:49.621146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1042/2068] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:51:49 2022] DELETE /identity/v3/projects/1d9a9c8b141844cd8f7c8a08f99b7bf8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:49.626414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.626964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.626964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.647684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:49.649020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:49.709881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1027/2069] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:49.717905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.718320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.718669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.741863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:49.743234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:49.752071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1043/2070] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:49.756261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.766521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c42e7b0-35b2-4a5b-a368-be6e4357a248 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.771145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.771732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.772006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.772275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.772929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:51:49.773214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.774882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.786922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1028/2071] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:51:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:51:49.791060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.801372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca80c7d-19ca-4749-af1f-a62697e88499 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.806609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.807317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.807617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.807887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.808629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.811176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.838151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1044/2072] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:49 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:49.844549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.855587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8caacd-bced-4f12-8b4d-9383a6314ffa admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.923456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.924345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.924861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.925020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.925650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.927674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.959019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1029/2073] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:49 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:49.960249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:49.971279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c9ba0-a6a4-4e13-b430-f21448a3086e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:49.976550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:49.977155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:49.977427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:49.977694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:49.978371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:49.980077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:49.999981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1045/2074] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:51:50.007435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.024989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37706d56-fdef-43f0-829c-a8d599b4a17d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.030929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda800f-fb39-45eb-a918-787c148551fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.031623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda800f-fb39-45eb-a918-787c148551fb admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.031902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda800f-fb39-45eb-a918-787c148551fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.032169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda800f-fb39-45eb-a918-787c148551fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.033105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda800f-fb39-45eb-a918-787c148551fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.034915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda800f-fb39-45eb-a918-787c148551fb admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.047981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1030/2075] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:50.052204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.073347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad3d6dd-3393-4b35-9e86-7180062504a5 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.079399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.080072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.080348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.080619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] PATH_INFO: `/v3/projects/5e4caf5c8afe4d07ad1c4c8700dbcd07/users/e1812470161b4af697ccee1733910807/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.096758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e4caf5c8afe4d07ad1c4c8700dbcd07, user_id=e1812470161b4af697ccee1733910807, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.099720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.115509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1046/2076] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:50 2022] PUT /identity/v3/projects/5e4caf5c8afe4d07ad1c4c8700dbcd07/users/e1812470161b4af697ccee1733910807/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:50.125600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.125918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.126213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.157811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] MFA Rules not processed for user `e1812470161b4af697ccee1733910807`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:50.159228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:50.212856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:50.236082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1031/2077] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:51:50.240545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.249724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:50.309895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c11b5-103a-405c-97dd-0e2d5ff55471 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.315168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.315780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.316049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.316321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.316971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.318906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.357968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1047/2078] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:51:50 2022] POST /identity/v3/projects => generated 377 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:50.361985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.373071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5eb99-44ec-4d44-a3b8-5f5a2f324c1d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.429337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.429963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.430239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.430504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.431141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.433027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.457780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1032/2079] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:51:50 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:51:50.461457 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.472998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5720b384-6f96-4eb4-a999-30f38dc6f1f4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.477990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.478681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.479023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.479359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.480065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.481703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.492406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1048/2080] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:50.497162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.507563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8a7073-1b21-4f8c-b83b-65443d2382af admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.513083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.513762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.514047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.514451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] PATH_INFO: `/v3/projects/5eac5cd98fdc458a9a592f515e2d8903/users/a74b639082cb4a9b90b4038d6507dd43/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.531230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eac5cd98fdc458a9a592f515e2d8903, user_id=a74b639082cb4a9b90b4038d6507dd43, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.534225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.563094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1033/2081] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:50 2022] PUT /identity/v3/projects/5eac5cd98fdc458a9a592f515e2d8903/users/a74b639082cb4a9b90b4038d6507dd43/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:50.582678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.599355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0fd7b0-f0ff-4317-accb-f5b085d2cd7c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.676535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2908e06d-4cf8-4603-b8f8-56b28c03129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.677167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e06d-4cf8-4603-b8f8-56b28c03129f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.677449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e06d-4cf8-4603-b8f8-56b28c03129f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.677757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2908e06d-4cf8-4603-b8f8-56b28c03129f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.678441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908e06d-4cf8-4603-b8f8-56b28c03129f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.680085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2908e06d-4cf8-4603-b8f8-56b28c03129f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.690771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1049/2082] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:51:50 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:51:50.694848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.705811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7ecba-68be-45a8-a02f-881f099423ba admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.710505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.711182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.711457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.711723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] PATH_INFO: `/v3/projects/5eac5cd98fdc458a9a592f515e2d8903/users/a74b639082cb4a9b90b4038d6507dd43/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.713608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5eac5cd98fdc458a9a592f515e2d8903, user_id=a74b639082cb4a9b90b4038d6507dd43, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:50.716313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:50.728550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1034/2083] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:51:50 2022] PUT /identity/v3/projects/5eac5cd98fdc458a9a592f515e2d8903/users/a74b639082cb4a9b90b4038d6507dd43/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:51:50.740464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.740756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.741058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:50.773792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] MFA Rules not processed for user `a74b639082cb4a9b90b4038d6507dd43`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:51:50.775186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:50.870416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1050/2084] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:51:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:51:50.884663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:51:50.886798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:51:50.950412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e1efbd-0e12-432d-bd85-1f8d93c63cd3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:51:50.953981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:51:50.954583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:51:50.954857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:51:50.955120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:51:51.018080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:51:51.022644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:51:51.024193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:51:51.034273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1035/2085] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:51:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:00.866455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:00.883170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:00.892528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac56d89-45aa-4d75-87be-ff2ba4eb6926 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:00.896103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:00.896697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:00.896969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:00.897234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:00.906576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:00.908241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:00.918061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1051/2086] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:01.029435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:10.913688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:23.330028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:23.331456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:23.412664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18230429-01e2-429b-9a82-dd46b1bc5369 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:23.417859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:23.418509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.418857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.419619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] PATH_INFO: `/v3/users/0792c6c88f9c4534aad5bf7ea3a4ac60` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:23.420854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0792c6c88f9c4534aad5bf7ea3a4ac60)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:23.422903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:23.469779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0792c6c88f9c4534aad5bf7ea3a4ac60'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.476809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0792c6c88f9c4534aad5bf7ea3a4ac60'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.477437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0792c6c88f9c4534aad5bf7ea3a4ac60'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.478040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0792c6c88f9c4534aad5bf7ea3a4ac60'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.486032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-443e8771-9aa0-4732-b9ab-77bc413ea14b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0792c6c88f9c4534aad5bf7ea3a4ac60'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.502773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1036/2087] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:23 2022] DELETE /identity/v3/users/0792c6c88f9c4534aad5bf7ea3a4ac60 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:23.677152 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:23.688305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021134f8-b530-467d-af7c-7d895d4dc591 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:23.744044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:23.744664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.744940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.745208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] PATH_INFO: `/v3/projects/ccf257bf6ccd476b839dbe55c8b27d4b` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:23.746307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf257bf6ccd476b839dbe55c8b27d4b)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:23.748238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:23.793201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf257bf6ccd476b839dbe55c8b27d4b', 'request_id': 'req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.799118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf257bf6ccd476b839dbe55c8b27d4b', 'request_id': 'req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.806019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf257bf6ccd476b839dbe55c8b27d4b', 'request_id': 'req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.816126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] The token cache is being invalidate because project ccf257bf6ccd476b839dbe55c8b27d4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:23.816415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.817048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dec25c7a-35da-4a6d-a907-aa6f2bc5ef2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:23.818771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1052/2088] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:23 2022] DELETE /identity/v3/projects/ccf257bf6ccd476b839dbe55c8b27d4b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:23.823620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.823906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.824181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:23.845482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:23.846318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:23.902658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1037/2089] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:23.917283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.917596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.917913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:23.937407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:23.938614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:23.946406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1053/2090] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:23.951086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:23.960598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168e4f8-8a9d-4bcf-9e4e-65a0df66c37e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:23.965034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:23.966249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.966528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.966794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:23.967432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:23.967728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:23.969296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:23.980503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1038/2091] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:23.984674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:23.993793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c34cbb-9e8d-4117-bf2b-6ea44db2393c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:23.998468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:23.999054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:23.999333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:23.999604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.000226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:24.002231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:24.030225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1054/2092] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:23 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:24.034482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:24.044326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2edaaa5-d40e-4995-8d68-b281856e2e39 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:24.047819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:24.101936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:24.102533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:24.102810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:24.103080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.103700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:24.105595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:24.126457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1039/2093] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:24 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:24.130513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:24.141456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764c995c-e468-4165-b99e-465e99cdd250 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:24.146075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:24.146672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:24.146948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:24.147219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.147852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:24.149433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:24.159475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1055/2094] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:24.163262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:24.173066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59e9e0-94d9-4477-9a63-3393235bc868 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:24.177666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14820d-907b-4444-bec6-c40fdfcd1f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:24.178281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14820d-907b-4444-bec6-c40fdfcd1f77 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:24.178559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14820d-907b-4444-bec6-c40fdfcd1f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:24.178856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14820d-907b-4444-bec6-c40fdfcd1f77 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.179494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14820d-907b-4444-bec6-c40fdfcd1f77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:24.181053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14820d-907b-4444-bec6-c40fdfcd1f77 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:24.191984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1040/2095] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:24.195731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:24.206596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aadf4-7946-4483-b258-e405f6f22127 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:24.211174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:24.211806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:24.212083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:24.212354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] PATH_INFO: `/v3/projects/1c5fb3ff52124db488fc164b595503c4/users/6a68278c8556492497def971fb1674a8/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.227370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c5fb3ff52124db488fc164b595503c4, user_id=6a68278c8556492497def971fb1674a8, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:24.230043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:24.242407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1056/2096] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:24 2022] PUT /identity/v3/projects/1c5fb3ff52124db488fc164b595503c4/users/6a68278c8556492497def971fb1674a8/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:24.247842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c0b77-3244-4932-89f0-b5d40844fe2a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:24.248124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c0b77-3244-4932-89f0-b5d40844fe2a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:24.248397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507c0b77-3244-4932-89f0-b5d40844fe2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.267465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-507c0b77-3244-4932-89f0-b5d40844fe2a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:24.268663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-507c0b77-3244-4932-89f0-b5d40844fe2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:24.328832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1041/2097] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:24.338627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:24.340327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:24.369937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:24.398914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93afa6c-e6b5-4450-81ba-8df680f2fbcb admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:24.402517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:24.403155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:24.403456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:24.403790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:24.413214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:24.414734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:24.424107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1057/2098] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:29.244590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf277-446f-42b7-8982-8e049846d98b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:29.245057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf277-446f-42b7-8982-8e049846d98b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:29.245137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1bf277-446f-42b7-8982-8e049846d98b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.292521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-de1bf277-446f-42b7-8982-8e049846d98b None None] MFA Rules not processed for user `6a68278c8556492497def971fb1674a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:29.294330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-de1bf277-446f-42b7-8982-8e049846d98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:29.386555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1042/2099] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:29.400827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:29.402330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:29.536051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8d1eea-06e0-47f0-83c2-5304bf4e8bb2 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:29.542074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fa5500-053b-41f9-a3c9-53b00e3299b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:29.542729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5500-053b-41f9-a3c9-53b00e3299b5 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:29.543232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5500-053b-41f9-a3c9-53b00e3299b5 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:29.543641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fa5500-053b-41f9-a3c9-53b00e3299b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:29.558975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5500-053b-41f9-a3c9-53b00e3299b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:29.561355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fa5500-053b-41f9-a3c9-53b00e3299b5 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:29.573920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1058/2100] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:30.886006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:30.886512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:30.886664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:30.905897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] MFA Rules not processed for user `6a68278c8556492497def971fb1674a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:30.907360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:30.914632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1043/2101] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:52:39.569506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=48 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:40.900251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:45.397812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91adb54-eaa3-4270-8604-7f641b5838b8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.398655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91adb54-eaa3-4270-8604-7f641b5838b8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.398932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91adb54-eaa3-4270-8604-7f641b5838b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.422585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b91adb54-eaa3-4270-8604-7f641b5838b8 None None] MFA Rules not processed for user `ac0e2925e78c4c35b116c8d0e03f576e`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:45.424111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b91adb54-eaa3-4270-8604-7f641b5838b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:45.504577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1059/2102] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:45.867710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:45.869237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:45.883688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68cffb6-a10f-4d99-918b-425db3bf993d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:45.888944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:45.889568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.889887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.890162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] PATH_INFO: `/v3/users/e1812470161b4af697ccee1733910807` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:45.891289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1812470161b4af697ccee1733910807)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:45.893241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:45.942393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1812470161b4af697ccee1733910807'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:45.950004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1812470161b4af697ccee1733910807'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:45.951311 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1812470161b4af697ccee1733910807'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:45.951910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1812470161b4af697ccee1733910807'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:45.959051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1812470161b4af697ccee1733910807'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:45.977447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1044/2103] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:45 2022] DELETE /identity/v3/users/e1812470161b4af697ccee1733910807 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:45.982193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:45.982554 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:45.982887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.002681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:46.003991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:46.062938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1060/2104] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:46.074001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.075954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:46.137639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c06b3-91df-43ee-901d-aaddc624654a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.142462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.143064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.143335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.143598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.155918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.157552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.168621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1045/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:46.520265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.521977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:46.534971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3bb30-db2f-4a22-9889-aa0d848cd116 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.540987 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.542000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.542565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.542989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] PATH_INFO: `/v3/projects/5e4caf5c8afe4d07ad1c4c8700dbcd07` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.575205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e4caf5c8afe4d07ad1c4c8700dbcd07)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.578197 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.643066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e4caf5c8afe4d07ad1c4c8700dbcd07', 'request_id': 'req-82c03474-b21b-4e1c-a58e-6f3be061d4dd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.649143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e4caf5c8afe4d07ad1c4c8700dbcd07', 'request_id': 'req-82c03474-b21b-4e1c-a58e-6f3be061d4dd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.657602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e4caf5c8afe4d07ad1c4c8700dbcd07', 'request_id': 'req-82c03474-b21b-4e1c-a58e-6f3be061d4dd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.670402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] The token cache is being invalidate because project 5e4caf5c8afe4d07ad1c4c8700dbcd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:46.670685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.671311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.672776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1061/2106] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:46 2022] DELETE /identity/v3/projects/5e4caf5c8afe4d07ad1c4c8700dbcd07 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:46.676897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:46.686403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea58455-1d55-4ee1-a0b3-146ca50fb45d service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:46.741213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:46.741866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:46.742141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:46.742409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] PATH_INFO: `/v3/users/a74b639082cb4a9b90b4038d6507dd43` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:46.743514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a74b639082cb4a9b90b4038d6507dd43)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:46.745428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:46.762370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:46.795024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a74b639082cb4a9b90b4038d6507dd43'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.803800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a74b639082cb4a9b90b4038d6507dd43'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.804446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a74b639082cb4a9b90b4038d6507dd43'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.805059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a74b639082cb4a9b90b4038d6507dd43'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.812502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a2a60b68-80d3-427c-bf0c-3e48d996e16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a74b639082cb4a9b90b4038d6507dd43'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:46.829950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1046/2107] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:46 2022] DELETE /identity/v3/users/a74b639082cb4a9b90b4038d6507dd43 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:47.094508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.096267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:47.110067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c03474-b21b-4e1c-a58e-6f3be061d4dd admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.136051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:47.169675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.170337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.170614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.170882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] PATH_INFO: `/v3/projects/5eac5cd98fdc458a9a592f515e2d8903` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.171990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5eac5cd98fdc458a9a592f515e2d8903)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.173997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.224771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5eac5cd98fdc458a9a592f515e2d8903', 'request_id': 'req-04a10228-522c-4728-84a2-5fd4b1db4638'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:47.236047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5eac5cd98fdc458a9a592f515e2d8903', 'request_id': 'req-04a10228-522c-4728-84a2-5fd4b1db4638'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:47.244820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5eac5cd98fdc458a9a592f515e2d8903', 'request_id': 'req-04a10228-522c-4728-84a2-5fd4b1db4638'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:47.259550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] The token cache is being invalidate because project 5eac5cd98fdc458a9a592f515e2d8903 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:47.259972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:47.260756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-04a10228-522c-4728-84a2-5fd4b1db4638 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:47.262437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1062/2108] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:47 2022] DELETE /identity/v3/projects/5eac5cd98fdc458a9a592f515e2d8903 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:47.267264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.267813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.267813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.289028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:47.290264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:47.348775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1047/2109] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:47.357369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.357652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.357952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.379208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:47.380656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:47.389495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1063/2110] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:47.393911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.404168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932cbc93-23b6-4720-886d-d5bf32bcef45 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.408795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.409645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.409951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.410221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.410883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:52:47.411182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.412750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.424667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1048/2111] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:52:47.428664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.437794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b648e0c6-8396-4ed9-9622-ce2041c02053 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.442483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.443068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.443346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.443616 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.444247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.446240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.477877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1064/2112] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/projects => generated 409 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:52:47.482574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.494478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7220bca-8b6d-4ed6-9be1-ac84e419f28a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.550795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.551490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.551792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.552054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.552713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.554690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.576011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1049/2113] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/users => generated 305 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:52:47.580129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.591470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb72da2-7c0d-43f7-9c1c-01c40b3b8bff admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.596666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.597282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.597592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.597897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.598548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.600139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.611340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1065/2114] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:47.615370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.625462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74016837-922a-4407-8707-9b22d71502f4 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.630698 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2932f6-f455-4d39-899f-5a8766da30cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.631284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2932f6-f455-4d39-899f-5a8766da30cf admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.631555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2932f6-f455-4d39-899f-5a8766da30cf admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.631823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2932f6-f455-4d39-899f-5a8766da30cf admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.632490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2932f6-f455-4d39-899f-5a8766da30cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.634087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2932f6-f455-4d39-899f-5a8766da30cf admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.639464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:47.645645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1050/2115] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:47.649834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.660821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7583a08-54c0-4b8c-a90b-1c8af216ecd6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.666638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.667300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.667576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.667848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] PATH_INFO: `/v3/projects/f8fb54b84c674dffb573e8b94fdaf77e/users/28b088045a5d405db95d286a58a249cc/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.684143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fb54b84c674dffb573e8b94fdaf77e, user_id=28b088045a5d405db95d286a58a249cc, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.687737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.701920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1066/2116] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:47 2022] PUT /identity/v3/projects/f8fb54b84c674dffb573e8b94fdaf77e/users/28b088045a5d405db95d286a58a249cc/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:47.713225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.713892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.714164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.744125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] MFA Rules not processed for user `28b088045a5d405db95d286a58a249cc`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:47.745536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:47.828362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1051/2117] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:47.832944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.891551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:47.897355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c9887a-3626-4e59-aad8-88f2be27c7f8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:47.903003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:47.904243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:47.904518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:47.904788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:47.905406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:47.907306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:47.934203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1067/2118] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/projects => generated 407 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:47.938302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:47.949100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54b91b-4e5a-4e0a-a1e8-ed3b3ae9258c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.010160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.010789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.011112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.011314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.011944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.014606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.050147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1052/2119] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:52:47 2022] POST /identity/v3/users => generated 310 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:52:48.054180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.065384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139716f-1f44-40bf-ab95-4b08ca9a7dcf admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.070221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.070816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.071096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.071365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.071999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.073588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.084624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1068/2120] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:48.088672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.099234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d959d91a-6c28-4046-af95-07a7c0651af4 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.103834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.104470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.104744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.105010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] PATH_INFO: `/v3/projects/844aaa3f63984888ab65c66b7fd4f5a9/users/b19b17476c1742f89f44b8b702af4346/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.119726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] RBAC: Authorizing `identity:create_grant(project_id=844aaa3f63984888ab65c66b7fd4f5a9, user_id=b19b17476c1742f89f44b8b702af4346, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.122926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.135849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1053/2121] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:48 2022] PUT /identity/v3/projects/844aaa3f63984888ab65c66b7fd4f5a9/users/b19b17476c1742f89f44b8b702af4346/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:48.140222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.150309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c06a3-842b-4c0a-82a2-87112ba6a1ee admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.206242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790d1ff-48ce-4866-9c14-3520a0f9346b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.206886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790d1ff-48ce-4866-9c14-3520a0f9346b admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.207155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790d1ff-48ce-4866-9c14-3520a0f9346b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.207418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790d1ff-48ce-4866-9c14-3520a0f9346b admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.208041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790d1ff-48ce-4866-9c14-3520a0f9346b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.209619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790d1ff-48ce-4866-9c14-3520a0f9346b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.219338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1069/2122] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:52:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:52:48.223447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.233644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df074c6-ee24-4e4e-bf97-b60591f8110f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.238207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.238848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.239119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.239383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] PATH_INFO: `/v3/projects/844aaa3f63984888ab65c66b7fd4f5a9/users/b19b17476c1742f89f44b8b702af4346/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.241236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844aaa3f63984888ab65c66b7fd4f5a9, user_id=b19b17476c1742f89f44b8b702af4346, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.243859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.254274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1054/2123] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:52:48 2022] PUT /identity/v3/projects/844aaa3f63984888ab65c66b7fd4f5a9/users/b19b17476c1742f89f44b8b702af4346/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:48.264837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf895-a6af-436f-8196-177d9068300e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.265120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf895-a6af-436f-8196-177d9068300e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.265385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280bf895-a6af-436f-8196-177d9068300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.293087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-280bf895-a6af-436f-8196-177d9068300e None None] MFA Rules not processed for user `b19b17476c1742f89f44b8b702af4346`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:48.294329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-280bf895-a6af-436f-8196-177d9068300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:48.365144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1070/2124] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:48.376085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.378059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:48.427235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:48.439965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63673bea-ded1-4690-b4d1-3194b8de3508 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.443699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.444290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.444558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.444819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.454727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.457189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.468304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1055/2125] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:48.634413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf895-a6af-436f-8196-177d9068300e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:48.644405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280bf895-a6af-436f-8196-177d9068300e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:48.648059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11970a7d-8d8b-424e-a7e4-6cb5cd406729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:48.648651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11970a7d-8d8b-424e-a7e4-6cb5cd406729 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:48.648932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11970a7d-8d8b-424e-a7e4-6cb5cd406729 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:48.649201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11970a7d-8d8b-424e-a7e4-6cb5cd406729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:48.696949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:48.708261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11970a7d-8d8b-424e-a7e4-6cb5cd406729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:48.709772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11970a7d-8d8b-424e-a7e4-6cb5cd406729 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:48.719138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1071/2126] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:49.074839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:49.076974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:49.142949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483642e-8599-4b10-b304-6ad993768ea3 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:49.148395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:49.148959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:49.149232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:49.149497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] PATH_INFO: `/v3/users/214e960097d248ba97e3d852d25cb075` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:49.150647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=214e960097d248ba97e3d852d25cb075)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:49.152759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:49.210963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '214e960097d248ba97e3d852d25cb075'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.219190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '214e960097d248ba97e3d852d25cb075'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.220003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '214e960097d248ba97e3d852d25cb075'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.220755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '214e960097d248ba97e3d852d25cb075'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.229962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '214e960097d248ba97e3d852d25cb075'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.248877 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1056/2127] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:49 2022] DELETE /identity/v3/users/214e960097d248ba97e3d852d25cb075 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:52:49.253382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:49.253761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:49.254176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:49.277724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:49.279709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:49.342126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1072/2128] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:49.351827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:49.353213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:49.416140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93a8d4-a0dd-4cce-86ff-2eeea05fba4d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:49.420226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:49.421178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:49.421178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:49.421406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:49.432024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:49.433594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:49.444893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1057/2129] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:49.883875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:49.885356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:49.899971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5543c-079e-47d5-98e7-509efb023bb4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:49.904599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:49.905226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:49.905865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:49.906146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] PATH_INFO: `/v3/projects/d31852d50f034c9a85bd20a8245bad38` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:49.907652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d31852d50f034c9a85bd20a8245bad38)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:49.909871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:49.955670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd31852d50f034c9a85bd20a8245bad38', 'request_id': 'req-81b77958-093e-4419-b986-e73977fb6ffe'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.961881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd31852d50f034c9a85bd20a8245bad38', 'request_id': 'req-81b77958-093e-4419-b986-e73977fb6ffe'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.969969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd31852d50f034c9a85bd20a8245bad38', 'request_id': 'req-81b77958-093e-4419-b986-e73977fb6ffe'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.981235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] The token cache is being invalidate because project d31852d50f034c9a85bd20a8245bad38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:49.982150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.982849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-81b77958-093e-4419-b986-e73977fb6ffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:49.984443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1073/2130] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:49 2022] DELETE /identity/v3/projects/d31852d50f034c9a85bd20a8245bad38 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:59.248889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:59.250322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:59.263114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4078b00-8cd8-4450-86f9-f591b43a0f50 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:59.266571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:59.320057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:59.320668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:59.320941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:59.321209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] PATH_INFO: `/v3/users/28b088045a5d405db95d286a58a249cc` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:59.322329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] RBAC: Authorizing `identity:delete_user(user_id=28b088045a5d405db95d286a58a249cc)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:59.324219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:59.369133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28b088045a5d405db95d286a58a249cc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.377493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28b088045a5d405db95d286a58a249cc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.378112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28b088045a5d405db95d286a58a249cc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.378729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28b088045a5d405db95d286a58a249cc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.386231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28b088045a5d405db95d286a58a249cc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.403600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1058/2131] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:59 2022] DELETE /identity/v3/users/28b088045a5d405db95d286a58a249cc => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:52:59.407982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:59.408440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:59.408553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:59.427488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:52:59.428750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:59.486081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1074/2132] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:52:59.496024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:59.498140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:59.554373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c100fe8-c613-4620-9fc4-60197548668a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:59.558146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:59.558740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:59.559018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:59.559768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:59.569937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:59.571364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:59.583309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1059/2133] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:52:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:52:59.866843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:59.868375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:52:59.881388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452042a7-a24f-4dff-8cb7-8b2ad2083e42 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:52:59.886368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:52:59.886996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:52:59.887273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:52:59.887682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] PATH_INFO: `/v3/projects/f8fb54b84c674dffb573e8b94fdaf77e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:52:59.889666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fb54b84c674dffb573e8b94fdaf77e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:52:59.891697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:52:59.900066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:52:59.940498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fb54b84c674dffb573e8b94fdaf77e', 'request_id': 'req-de100d78-2aa2-4af6-8079-08fd2654f6f2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.946524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fb54b84c674dffb573e8b94fdaf77e', 'request_id': 'req-de100d78-2aa2-4af6-8079-08fd2654f6f2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.953948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fb54b84c674dffb573e8b94fdaf77e', 'request_id': 'req-de100d78-2aa2-4af6-8079-08fd2654f6f2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.964136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] The token cache is being invalidate because project f8fb54b84c674dffb573e8b94fdaf77e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:52:59.964418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.965059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:52:59.966564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1075/2134] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:52:59 2022] DELETE /identity/v3/projects/f8fb54b84c674dffb573e8b94fdaf77e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:52:59.970779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:52:59.980128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53badf-ecda-4a98-a70e-5a9d1b049ac1 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.007903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:00.047082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.047719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.047994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.048270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] PATH_INFO: `/v3/users/b19b17476c1742f89f44b8b702af4346` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.049411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b19b17476c1742f89f44b8b702af4346)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.051351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:00.102136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b19b17476c1742f89f44b8b702af4346'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.110771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b19b17476c1742f89f44b8b702af4346'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.111501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b19b17476c1742f89f44b8b702af4346'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.112181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b19b17476c1742f89f44b8b702af4346'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.119968 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-05810a1d-d2a8-428a-ac4f-91d5ebd2c0f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b19b17476c1742f89f44b8b702af4346'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.138137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1060/2135] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:52:59 2022] DELETE /identity/v3/users/b19b17476c1742f89f44b8b702af4346 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:00.429358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:00.430971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:00.456236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de100d78-2aa2-4af6-8079-08fd2654f6f2 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.549155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.550239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.550876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.551496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] PATH_INFO: `/v3/projects/844aaa3f63984888ab65c66b7fd4f5a9` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.552764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] RBAC: Authorizing `identity:delete_project(project_id=844aaa3f63984888ab65c66b7fd4f5a9)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.555052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:00.605440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844aaa3f63984888ab65c66b7fd4f5a9', 'request_id': 'req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.611939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844aaa3f63984888ab65c66b7fd4f5a9', 'request_id': 'req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.622060 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844aaa3f63984888ab65c66b7fd4f5a9', 'request_id': 'req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.632997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] The token cache is being invalidate because project 844aaa3f63984888ab65c66b7fd4f5a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:00.633485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.634174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f1723fbe-edc8-47c4-a9f2-fe7fe8ba336e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:00.635655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1076/2136] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:00 2022] DELETE /identity/v3/projects/844aaa3f63984888ab65c66b7fd4f5a9 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:00.640938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.641426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.641609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.661822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:00.663052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:00.723158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1061/2137] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:00.732009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.732292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.732567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.752165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:00.753380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:00.761291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1077/2138] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:00.766102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:00.775608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741acb0-353d-4a1b-aa70-7af3082b0e2e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.780942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.781529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.781829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.782105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.782738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:53:00.783035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.784876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:00.796158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1062/2139] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:53:00.800344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:00.809465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd50521-27ca-41f6-8df5-1abd1400cf37 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.814075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.814656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.814931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.815200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.815826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.817699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:00.838552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:00.847615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1078/2140] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:00 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:00.851335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:00.861141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee19a7-2dfe-4261-a475-991fad88eaf2 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.893637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:00.920464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.921056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.921327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.921590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.922248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.924152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:00.944136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1063/2141] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:00 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:00.948118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:00.959082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3460c9b6-a20e-41b9-a4fe-ca55cde7032a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.963651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.964240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.964519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.964790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.965432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.967048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:00.976681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1079/2142] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:00.980247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:00.990536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d85b1-509f-4744-99fa-28d1576d4630 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:00.995107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce59072-a91f-4cfb-b2c5-c617e3f4c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:00.995684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59072-a91f-4cfb-b2c5-c617e3f4c02a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:00.995955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59072-a91f-4cfb-b2c5-c617e3f4c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:00.996220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce59072-a91f-4cfb-b2c5-c617e3f4c02a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:00.997360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce59072-a91f-4cfb-b2c5-c617e3f4c02a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:00.998935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce59072-a91f-4cfb-b2c5-c617e3f4c02a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.008552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1064/2143] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:01.013089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.023680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b72837-d053-46b0-bd27-782b12e7e00a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.028214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.028859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.029134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.029403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] PATH_INFO: `/v3/projects/586899bdabd74bb69e8c1cc1b09b6864/users/f1e64f3983a549cca5357445d9fd9833/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.044098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586899bdabd74bb69e8c1cc1b09b6864, user_id=f1e64f3983a549cca5357445d9fd9833, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.046827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.060648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1080/2144] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:01 2022] PUT /identity/v3/projects/586899bdabd74bb69e8c1cc1b09b6864/users/f1e64f3983a549cca5357445d9fd9833/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:01.071078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.071360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.071626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.099927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] MFA Rules not processed for user `f1e64f3983a549cca5357445d9fd9833`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:01.101128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:01.163444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1065/2145] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:01.167838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.228492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319643a-22ce-470c-9a3d-febbe27af5d1 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.233853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.234461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.234738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.235012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.235677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.237604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.269687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1081/2146] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:01 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:01.270821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.281711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bdc856-7569-4bf5-8feb-aa0189e39de5 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.339926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.340880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.340880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.341129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.341761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.343646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.366972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1066/2147] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:01 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:01.370796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.381452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9108918-e06c-4a88-88ef-3e7bdc05a207 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.386366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.386956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.387224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.387520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.388159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.389761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.400181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1082/2148] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:01.404235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.414252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25e8386-5ecf-4e02-aba6-d55c76b77b9e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.419262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.419894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.420168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.420437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] PATH_INFO: `/v3/projects/d7884a3be91c43c38ca0a1b7a2bdfdf7/users/d28204632d0f48da95dd6bf9fa6f614e/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.435211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7884a3be91c43c38ca0a1b7a2bdfdf7, user_id=d28204632d0f48da95dd6bf9fa6f614e, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.437900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.450447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1067/2149] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:01 2022] PUT /identity/v3/projects/d7884a3be91c43c38ca0a1b7a2bdfdf7/users/d28204632d0f48da95dd6bf9fa6f614e/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:01.454239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.464040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315b8ac6-eb67-408d-80ed-e103d5993a5f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.530538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be8258-5b53-4a8c-996c-e897a705994f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.531081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be8258-5b53-4a8c-996c-e897a705994f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.531365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be8258-5b53-4a8c-996c-e897a705994f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.531643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be8258-5b53-4a8c-996c-e897a705994f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.532404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be8258-5b53-4a8c-996c-e897a705994f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.534705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be8258-5b53-4a8c-996c-e897a705994f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.551485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1083/2150] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:01 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:01.555532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.569489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798e545d-944e-4030-a2c8-5ed120d7ffa5 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.586381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.587134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.587578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.587856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] PATH_INFO: `/v3/projects/d7884a3be91c43c38ca0a1b7a2bdfdf7/users/d28204632d0f48da95dd6bf9fa6f614e/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.590212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7884a3be91c43c38ca0a1b7a2bdfdf7, user_id=d28204632d0f48da95dd6bf9fa6f614e, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.595077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.604410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:01.616761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1068/2151] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:01 2022] PUT /identity/v3/projects/d7884a3be91c43c38ca0a1b7a2bdfdf7/users/d28204632d0f48da95dd6bf9fa6f614e/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:01.628276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.628954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.629517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.678160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] MFA Rules not processed for user `d28204632d0f48da95dd6bf9fa6f614e`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:01.679390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:01.704105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:01.772118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1084/2152] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:01.790224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:01.792841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:01.900008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0281d398-5deb-4e3b-99ff-12a9d499458c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:01.905975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:01.908004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:01.908513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:01.908891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:01.922866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:01.924526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:01.936218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1069/2153] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:02.059654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:02.071507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb378b-fbe5-40e9-8c74-518656042e30 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:02.076304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1d3dc0-fd9d-4cd0-854e-301880751c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:02.077056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3dc0-fd9d-4cd0-854e-301880751c47 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:02.077399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3dc0-fd9d-4cd0-854e-301880751c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:02.077762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1d3dc0-fd9d-4cd0-854e-301880751c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:02.150415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d3dc0-fd9d-4cd0-854e-301880751c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:02.151846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1d3dc0-fd9d-4cd0-854e-301880751c47 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:02.163131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1085/2154] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:02.426851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:02.486373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6d45e-a06e-474e-b3f6-cf1a518777c2 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:02.490763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:02.491372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:02.491644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:02.491911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] PATH_INFO: `/v3/users/f1e64f3983a549cca5357445d9fd9833` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:02.493007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1e64f3983a549cca5357445d9fd9833)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:02.494920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:02.540323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1e64f3983a549cca5357445d9fd9833'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:02.548229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1e64f3983a549cca5357445d9fd9833'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:02.548879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1e64f3983a549cca5357445d9fd9833'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:02.549512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1e64f3983a549cca5357445d9fd9833'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:02.556456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1e64f3983a549cca5357445d9fd9833'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:02.580517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1070/2155] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:02 2022] DELETE /identity/v3/users/f1e64f3983a549cca5357445d9fd9833 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:02.585080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:02.585569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:02.585674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:02.605866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:02.607058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:02.663721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1086/2156] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:02.673394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:02.674804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:02.695272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:02.734483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49519d83-c227-41b4-89af-dcd760b07749 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:02.738104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:02.738705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:02.739076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:02.739354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:02.749006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:02.750454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:02.759863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1071/2157] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:03.121358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:03.122950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.135046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c40330-a705-455e-90fc-1238e0de14f8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:03.139583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:03.140203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.140489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.140749 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] PATH_INFO: `/v3/projects/586899bdabd74bb69e8c1cc1b09b6864` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.141898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] RBAC: Authorizing `identity:delete_project(project_id=586899bdabd74bb69e8c1cc1b09b6864)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:03.143779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:03.200405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586899bdabd74bb69e8c1cc1b09b6864', 'request_id': 'req-0f3d3668-50c5-4687-9190-eb4e894d5edd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.208136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:03.213233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586899bdabd74bb69e8c1cc1b09b6864', 'request_id': 'req-0f3d3668-50c5-4687-9190-eb4e894d5edd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.222493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586899bdabd74bb69e8c1cc1b09b6864', 'request_id': 'req-0f3d3668-50c5-4687-9190-eb4e894d5edd'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.241458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] The token cache is being invalidate because project 586899bdabd74bb69e8c1cc1b09b6864 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:03.241779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.242844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.246012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1087/2158] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:03 2022] DELETE /identity/v3/projects/586899bdabd74bb69e8c1cc1b09b6864 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:03.248251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:03.258410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f2ba8-fd91-456d-a7f1-b5ab101adaa2 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:03.316182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:03.316804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.317074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.317339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] PATH_INFO: `/v3/users/d28204632d0f48da95dd6bf9fa6f614e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.318510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d28204632d0f48da95dd6bf9fa6f614e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:03.320426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:03.375166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd28204632d0f48da95dd6bf9fa6f614e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.383423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd28204632d0f48da95dd6bf9fa6f614e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.384165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd28204632d0f48da95dd6bf9fa6f614e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.385035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd28204632d0f48da95dd6bf9fa6f614e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.392893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93dba1c6-71d5-465f-81c6-c56f615345c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd28204632d0f48da95dd6bf9fa6f614e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.410681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1072/2159] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:03 2022] DELETE /identity/v3/users/d28204632d0f48da95dd6bf9fa6f614e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:03.677824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:03.679307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.694073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3668-50c5-4687-9190-eb4e894d5edd admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:03.750082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:03.750706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.750985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.751258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] PATH_INFO: `/v3/projects/d7884a3be91c43c38ca0a1b7a2bdfdf7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.752353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7884a3be91c43c38ca0a1b7a2bdfdf7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:03.754366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:03.799396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7884a3be91c43c38ca0a1b7a2bdfdf7', 'request_id': 'req-f558010c-254d-465c-89c5-e4bf084eae20'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.805801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7884a3be91c43c38ca0a1b7a2bdfdf7', 'request_id': 'req-f558010c-254d-465c-89c5-e4bf084eae20'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.813560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7884a3be91c43c38ca0a1b7a2bdfdf7', 'request_id': 'req-f558010c-254d-465c-89c5-e4bf084eae20'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.823701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] The token cache is being invalidate because project d7884a3be91c43c38ca0a1b7a2bdfdf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:03.823982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.824589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f558010c-254d-465c-89c5-e4bf084eae20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:03.829351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1088/2160] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:03 2022] DELETE /identity/v3/projects/d7884a3be91c43c38ca0a1b7a2bdfdf7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:03.835001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.835287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.835565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.856940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:03.858163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.889390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:03.916214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1073/2161] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:03.925513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.925830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.926104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.945372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:03.946595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:03.964255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1089/2162] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:03.968988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:03.978791 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d755d0-4956-4349-94f0-44e7d737f7df None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:03.983974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:03.984563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:03.984859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:03.985112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:03.985817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:53:03.986118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:03.987723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:03.999151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1074/2163] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:53:04.003394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.013385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a6742-50a8-4e45-a9fa-c74d1b9805bb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.019055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.019767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.020047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.020394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.021102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.023099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.055671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1090/2164] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:04 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:04.056588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.066683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f57fd96-ae64-4690-9b3c-58636ad1d7a3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.122921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.123522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.123798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.124068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.124688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.126607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.147764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1075/2165] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:04 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:04.151800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.162950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2274ba-c304-402d-9950-2d92f383ef31 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.167571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.168166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.168442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.169297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.169993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.171583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.181782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1091/2166] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:04.187282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.199000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bd0746-9c0c-4145-9e23-3b10f1880c6b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.204437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d59aa-4a10-4d12-90f2-be12541d1b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.205073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d59aa-4a10-4d12-90f2-be12541d1b56 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.205325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d59aa-4a10-4d12-90f2-be12541d1b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.205627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d59aa-4a10-4d12-90f2-be12541d1b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.206311 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d59aa-4a10-4d12-90f2-be12541d1b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.207894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d59aa-4a10-4d12-90f2-be12541d1b56 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.220205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1076/2167] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:53:04.224474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.235096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5773b6b1-a377-4b3d-a9ab-10c01a630a70 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.240081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.240321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.240598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.240892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] PATH_INFO: `/v3/projects/b1dfc7149bd64041bbeb877fe9767d9a/users/8010a18781df402ca16bf8bc9a88299a/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.255890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1dfc7149bd64041bbeb877fe9767d9a, user_id=8010a18781df402ca16bf8bc9a88299a, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.258620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.274487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1092/2168] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:04 2022] PUT /identity/v3/projects/b1dfc7149bd64041bbeb877fe9767d9a/users/8010a18781df402ca16bf8bc9a88299a/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:04.285443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fe463-4c48-4f72-b498-a94f8a88e81a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.286033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fe463-4c48-4f72-b498-a94f8a88e81a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.286033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59fe463-4c48-4f72-b498-a94f8a88e81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.351275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d59fe463-4c48-4f72-b498-a94f8a88e81a None None] MFA Rules not processed for user `8010a18781df402ca16bf8bc9a88299a`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:04.352508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d59fe463-4c48-4f72-b498-a94f8a88e81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:04.421107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1077/2169] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:04.432326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:04.434286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:04.444967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:04.497604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbfffd4-a9de-4b5f-ac3f-e3f5e2490a3a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:04.504531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad51268-7fcc-4b83-a526-f7af8b1f3bd7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:04.506021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad51268-7fcc-4b83-a526-f7af8b1f3bd7 service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:04.506595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad51268-7fcc-4b83-a526-f7af8b1f3bd7 service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:04.506960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad51268-7fcc-4b83-a526-f7af8b1f3bd7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:04.522196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad51268-7fcc-4b83-a526-f7af8b1f3bd7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:04.522196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad51268-7fcc-4b83-a526-f7af8b1f3bd7 service glance] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:04.535092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1093/2170] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:53:14.421780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:14.530656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=23 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:16.755227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:16.756277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:16.756750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:16.779097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] MFA Rules not processed for user `53fa55dda30e4aaa86ae599e4df2fed3`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:16.780291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:16.841317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1078/2171] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:16 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:25.288586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:25.289094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:25.289183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:25.312952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:25.312952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:25.379013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1094/2172] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 19:53:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:25.432517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:25.433958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:25.514778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f79662-c70a-452d-997d-d59e8b2d808c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:25.520066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:25.520943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:25.521282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:25.521705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:25.530882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:25.532960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:25.541419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1079/2173] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 19:53:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:53:33.377327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:33.378803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:33.482986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95effbf0-59ed-45bf-8e1d-a38180c5c340 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:33.488344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:33.488970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:33.489254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:33.489524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:33.491489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:33.504397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:33.506389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b712d2-f21b-4e71-9edf-088cc9a36f33 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:33.531000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1095/2174] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:35.538249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=57 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:39.305470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:39.388439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f728bc-a81b-4b40-b68b-22bdc170c2ce service swift] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:39.394061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:39.394944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:39.395569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:39.396180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] PATH_INFO: `/v3/users/8010a18781df402ca16bf8bc9a88299a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:39.397592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8010a18781df402ca16bf8bc9a88299a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:39.399741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:39.459498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8010a18781df402ca16bf8bc9a88299a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:39.471385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8010a18781df402ca16bf8bc9a88299a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:39.472019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8010a18781df402ca16bf8bc9a88299a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:39.472628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8010a18781df402ca16bf8bc9a88299a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:39.487077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8010a18781df402ca16bf8bc9a88299a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:39.510461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1080/2175] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:39 2022] DELETE /identity/v3/users/8010a18781df402ca16bf8bc9a88299a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:39.515044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:39.515559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:39.515661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:39.538739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:39.540187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:39.615487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1096/2176] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:39.625268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:39.626694 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:39.722034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef61fd8-0b87-43a6-9495-e441c1bb71dc admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:39.731243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a642e2-1ec0-436f-8293-6b5db3c8e020 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:39.732029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a642e2-1ec0-436f-8293-6b5db3c8e020 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:39.732405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a642e2-1ec0-436f-8293-6b5db3c8e020 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:39.732807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a642e2-1ec0-436f-8293-6b5db3c8e020 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:39.743317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a642e2-1ec0-436f-8293-6b5db3c8e020 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:39.744886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a642e2-1ec0-436f-8293-6b5db3c8e020 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:39.760729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1081/2177] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:40.254171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:40.255964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:40.290936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5eb0c-96a2-4f52-a14c-4dcfe6611a56 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:40.303628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:40.304280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.304558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.304834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] PATH_INFO: `/v3/projects/b1dfc7149bd64041bbeb877fe9767d9a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.306574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1dfc7149bd64041bbeb877fe9767d9a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:40.307920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:40.353113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:40.372844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1dfc7149bd64041bbeb877fe9767d9a', 'request_id': 'req-403c526b-63ce-484e-af95-caad8bd364dc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:40.380721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1dfc7149bd64041bbeb877fe9767d9a', 'request_id': 'req-403c526b-63ce-484e-af95-caad8bd364dc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:40.390570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1dfc7149bd64041bbeb877fe9767d9a', 'request_id': 'req-403c526b-63ce-484e-af95-caad8bd364dc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:40.403508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] The token cache is being invalidate because project b1dfc7149bd64041bbeb877fe9767d9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:40.403834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:40.405781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-403c526b-63ce-484e-af95-caad8bd364dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:40.411670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1097/2178] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:40 2022] DELETE /identity/v3/projects/b1dfc7149bd64041bbeb877fe9767d9a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:40.412420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.412846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.413007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.450573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:40.452184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:40.474358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:40.627713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1082/2179] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:40.629698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.630293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.630929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.655118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:40.656484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:40.677802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1098/2180] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:40.679668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:40.692691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bf7dd0-b494-45ee-9ad9-3dcd89d646d9 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:40.700913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:40.701900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.702348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.702773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.703702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:53:40.704365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:40.706200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:40.721969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1083/2181] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:53:40.729839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:40.737829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ced1-c4e9-40e4-a18e-8c6af708bd2c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:40.743730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:40.744666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.745114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.745630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.746524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:40.749943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:40.791525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1099/2182] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:40 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:40.793577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:40.804733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d6d571-ec72-4eb9-aee5-6b695bc6ecc5 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:40.881605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:40.882268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.882546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.882818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.883506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:40.885705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:40.917029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1084/2183] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:40 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:40.918223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:40.928924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14430d2a-1e56-42ed-9e5f-a7d06332a8c5 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:40.933878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:40.934486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.934759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.935029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.935706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:40.937306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:40.952198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1100/2184] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:40.953695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:40.964680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e340a8-2287-45b9-b7ca-1a63194fa646 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:40.969998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0d11d-85cb-4fee-bca9-5d1ed57d7e1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:40.970605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0d11d-85cb-4fee-bca9-5d1ed57d7e1e admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:40.970892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0d11d-85cb-4fee-bca9-5d1ed57d7e1e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:40.971181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0d11d-85cb-4fee-bca9-5d1ed57d7e1e admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:40.971874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0d11d-85cb-4fee-bca9-5d1ed57d7e1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:40.973651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0d11d-85cb-4fee-bca9-5d1ed57d7e1e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:40.989003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1085/2185] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:40.990199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.000464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c78df-7619-4521-8591-adb27c41f20c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.006957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.007839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.008068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.008429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] PATH_INFO: `/v3/projects/ad90199955814e0c93040aaade3bf506/users/b04c9f0b363a4c49b4a2d1a253691b2e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.026865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad90199955814e0c93040aaade3bf506, user_id=b04c9f0b363a4c49b4a2d1a253691b2e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.030324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:41.047034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1101/2186] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:40 2022] PUT /identity/v3/projects/ad90199955814e0c93040aaade3bf506/users/b04c9f0b363a4c49b4a2d1a253691b2e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:41.058718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.059176 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.059517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.096448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] MFA Rules not processed for user `b04c9f0b363a4c49b4a2d1a253691b2e`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:41.097944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:41.187879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1086/2187] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:41.191981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.315597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a264a-0d01-4c28-9878-034449fa44fc admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.324370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.325321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.325602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.325914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.327102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.329020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:41.379667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1102/2188] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:41 2022] POST /identity/v3/projects => generated 385 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:41.384254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.396830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cc7a09-ba26-4c38-aea8-c2208b2184e7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.411761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:41.477846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.478563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.478965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.481084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.484708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.484708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:41.515086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1087/2189] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:41 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:41.518874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.532357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc990f-87a9-4658-99ef-fe099f5bf1e4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.543312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.544806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.546184 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.546565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.549020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.555329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:41.582426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1103/2190] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:41.584168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.596184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70a00d-795e-439d-9cf7-b7d56ffeaf64 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.601539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e48af8-3cf6-4c3f-9079-e00603be719c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.602237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e48af8-3cf6-4c3f-9079-e00603be719c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.602513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e48af8-3cf6-4c3f-9079-e00603be719c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.602784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e48af8-3cf6-4c3f-9079-e00603be719c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.604148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e48af8-3cf6-4c3f-9079-e00603be719c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.605904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e48af8-3cf6-4c3f-9079-e00603be719c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:41.622917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1088/2191] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:41.626021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.636669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a587bb-4b4c-4df7-bb90-19df37597b05 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.644172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.645045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.645386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.645754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] PATH_INFO: `/v3/projects/4662a6c75e174389adfe542182e25580/users/a06a1b2930cf4e3788c0414a8a9c46d5/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.672321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] RBAC: Authorizing `identity:create_grant(project_id=4662a6c75e174389adfe542182e25580, user_id=a06a1b2930cf4e3788c0414a8a9c46d5, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.676391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:41.709041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1104/2192] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:41 2022] PUT /identity/v3/projects/4662a6c75e174389adfe542182e25580/users/a06a1b2930cf4e3788c0414a8a9c46d5/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:53:41.709675 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.710070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.717773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.776208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] MFA Rules not processed for user `a06a1b2930cf4e3788c0414a8a9c46d5`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:41.782682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:41.886885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1089/2193] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:41.891664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:41.898168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:41.979055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b9df33-17fd-4269-9e72-c98eed09baba admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:41.985441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:41.986361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:41.986640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:41.986913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:41.987703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:41.989800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.024970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1105/2194] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:41 2022] POST /identity/v3/projects => generated 385 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:42.026447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.038818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038a67c-ae1c-483b-98ff-4c94dbe1f336 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.114955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.115567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.115840 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.116100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.116993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.119103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.144065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1090/2195] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:42 2022] POST /identity/v3/users => generated 299 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:42.148797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.160758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98547ab3-c040-4e66-8650-8be6563f0b93 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.165718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.166349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.166627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.166900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.167541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.169131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.180844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1106/2196] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:42.184865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.195150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1eb71-6461-41b7-bc0a-cf78d1b25156 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.199974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.200874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.201183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.201461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] PATH_INFO: `/v3/projects/73d57859e9b8410bb057723a9efe6fcc/users/7356c58b41784addaba03f0693308625/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.214137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:42.217953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d57859e9b8410bb057723a9efe6fcc, user_id=7356c58b41784addaba03f0693308625, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.221932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.240251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1091/2197] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:42 2022] PUT /identity/v3/projects/73d57859e9b8410bb057723a9efe6fcc/users/7356c58b41784addaba03f0693308625/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:42.244011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.254936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746b828-06e3-4e49-b692-871a77c3381f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.313082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f16bb-be9b-4358-844f-0e550a2250fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.313685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f16bb-be9b-4358-844f-0e550a2250fc admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.313991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f16bb-be9b-4358-844f-0e550a2250fc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.314289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f16bb-be9b-4358-844f-0e550a2250fc admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.314921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f16bb-be9b-4358-844f-0e550a2250fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.316502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f16bb-be9b-4358-844f-0e550a2250fc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.327304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1107/2198] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:42 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:42.331634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.344829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f540ccd-48a5-4307-b281-4b7e6e4f44a9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.349470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.350149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.350506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.350777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] PATH_INFO: `/v3/projects/73d57859e9b8410bb057723a9efe6fcc/users/7356c58b41784addaba03f0693308625/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.353218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73d57859e9b8410bb057723a9efe6fcc, user_id=7356c58b41784addaba03f0693308625, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.355870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.368254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1092/2199] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:42 2022] PUT /identity/v3/projects/73d57859e9b8410bb057723a9efe6fcc/users/7356c58b41784addaba03f0693308625/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:42.379413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac802-2c0e-45c7-98f2-86b0e9c9318c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.379767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac802-2c0e-45c7-98f2-86b0e9c9318c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.380119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8ac802-2c0e-45c7-98f2-86b0e9c9318c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.417079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-af8ac802-2c0e-45c7-98f2-86b0e9c9318c None None] MFA Rules not processed for user `7356c58b41784addaba03f0693308625`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:42.418409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-af8ac802-2c0e-45c7-98f2-86b0e9c9318c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:42.486363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:42.490550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1108/2200] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:42 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:42.501949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.504202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:42.569225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce2d17f-45c4-4e53-b834-cbaa24fe6172 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.573517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.574267 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.574641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.575006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.613302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.613593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.613952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.633366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:42.634641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:42.640505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.642116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.651751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1093/2201] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:42.700602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1109/2202] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 19:53:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:53:42.716193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:42.718575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:42.808596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b559da-0a30-49e8-865e-3e7e3aa958c0 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:42.814475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:42.815082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:42.815486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:42.815708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:42.829825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:42.833825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:42.846710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1094/2203] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 19:53:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 19:53:45.272156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:45.273556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:45.285350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e13821-d0c9-498e-98cd-2e33db713bd4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:45.288720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa57f1-c4e3-4269-af42-783640ddd3fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:45.289305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa57f1-c4e3-4269-af42-783640ddd3fb service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:45.289577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa57f1-c4e3-4269-af42-783640ddd3fb service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:45.289876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa57f1-c4e3-4269-af42-783640ddd3fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:45.344337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa57f1-c4e3-4269-af42-783640ddd3fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:45.345779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aa57f1-c4e3-4269-af42-783640ddd3fb service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:45.354923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1110/2204] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:45.949433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:45.962526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a99f0b-e3d8-4db6-a977-eafa8106577f None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:45.962813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a99f0b-e3d8-4db6-a977-eafa8106577f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:45.963091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a99f0b-e3d8-4db6-a977-eafa8106577f None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:45.963554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0226fbd6-dd82-49ab-a3ba-9ef15320379d service swift] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:45.965351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1111/2205] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 19:53:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 19:53:45.968940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:45.969696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:45.970021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:45.970346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:45.980809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:45.982452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:45.992654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1095/2206] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:45 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:46.015300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.015706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.016080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.040501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.042002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] MFA Rules not processed for user `1f3be656bf6446f99a23910cd833d799`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:46.042295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:46.043155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:46.055829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:46.121627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1112/2207] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 19:53:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:46.128428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47260a8a-5765-4070-8c90-5218a62e64b9 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.132938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.133225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.134003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.134280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.134386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:46.134551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] PATH_INFO: `/v3/users/ac0e2925e78c4c35b116c8d0e03f576e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.135688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0e2925e78c4c35b116c8d0e03f576e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.137597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.148400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d85e36-4a2d-4747-87d8-7daf2f2bdc6e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.153618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.154284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.154574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.154841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.164321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.166533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.179123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1113/2208] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 19:53:46.188665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0e2925e78c4c35b116c8d0e03f576e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.196485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0e2925e78c4c35b116c8d0e03f576e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.197389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0e2925e78c4c35b116c8d0e03f576e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.198035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0e2925e78c4c35b116c8d0e03f576e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.205787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a4d6926d-b71d-4bbd-8435-dd25c1d9efed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0e2925e78c4c35b116c8d0e03f576e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.223600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1096/2209] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:46 2022] DELETE /identity/v3/users/ac0e2925e78c4c35b116c8d0e03f576e => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:53:46.406465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.419399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb02ca9-09f3-4283-bcbf-b4c06227858d service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.445923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:46.477756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.478392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.478668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.478938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] PATH_INFO: `/v3/projects/fb453c35c90c4c578f97a6fa48b8f710` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.480028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb453c35c90c4c578f97a6fa48b8f710)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.481973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.527127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb453c35c90c4c578f97a6fa48b8f710', 'request_id': 'req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.533835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb453c35c90c4c578f97a6fa48b8f710', 'request_id': 'req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.541836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb453c35c90c4c578f97a6fa48b8f710', 'request_id': 'req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.553291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] The token cache is being invalidate because project fb453c35c90c4c578f97a6fa48b8f710 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:46.553577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.554262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c99dc23d-fd8c-41ce-aa4a-2aa99ab2968a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:46.555756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1114/2210] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:46 2022] DELETE /identity/v3/projects/fb453c35c90c4c578f97a6fa48b8f710 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:46.560881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.561196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.561472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.581571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:46.582833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:46.641046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1097/2211] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:46.648713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.649001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.649276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.669609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:46.671453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:46.679524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1115/2212] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:46.683639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.693300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a8cb7-8264-4472-a6eb-7129beb22245 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.697886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.698475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.698756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.699031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.699672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:53:46.699976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.701559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.713172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1098/2213] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:53:46.718126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.727624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170af41-9ba1-486a-aff8-a807c40d2ad0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.732281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.733458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.733754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.734040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.734672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.736565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.768041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1116/2214] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:46 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:46.769167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.779040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1119e1-887a-4ef6-bdae-22392824f817 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.791127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:46.836997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.837546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.837857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.838124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.838799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.841044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.863746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1099/2215] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:46 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:53:46.868038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.878220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327ab0c-4c9a-4683-a491-6c7fb320a742 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.882770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.883352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.883627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.883892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.884524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.886179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.896319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1117/2216] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:46.900124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.910925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c240fe-ec94-4a5f-8609-879d3d0c1335 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.915492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816d874-ad18-42ba-bc0a-647f7dfbe281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.916071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d874-ad18-42ba-bc0a-647f7dfbe281 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.916347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d874-ad18-42ba-bc0a-647f7dfbe281 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.916615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816d874-ad18-42ba-bc0a-647f7dfbe281 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.917248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816d874-ad18-42ba-bc0a-647f7dfbe281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.918841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816d874-ad18-42ba-bc0a-647f7dfbe281 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.929268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1100/2217] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:46.933106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:46.942833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22707b-95f0-49fe-a6b9-a06aa561e137 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:46.947389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:46.948020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.948294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.948566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] PATH_INFO: `/v3/projects/8596f8b524fd417986d83d2d4b3210d1/users/30e37aad5ec141e49a4aea056c17f063/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:46.963568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8596f8b524fd417986d83d2d4b3210d1, user_id=30e37aad5ec141e49a4aea056c17f063, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:46.966218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:46.979523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1118/2218] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:46 2022] PUT /identity/v3/projects/8596f8b524fd417986d83d2d4b3210d1/users/30e37aad5ec141e49a4aea056c17f063/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:46.989702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:46.990203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:46.990309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:47.017264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] MFA Rules not processed for user `30e37aad5ec141e49a4aea056c17f063`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:47.018487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:47.081199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1101/2219] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:47.093415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:47.095097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:47.153312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf70b6d-8f3a-4657-9fdc-03dd9b65bd69 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:47.158057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8f0f0a-e397-444e-b830-9c144ff1672e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:47.158720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0f0a-e397-444e-b830-9c144ff1672e service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:47.159005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0f0a-e397-444e-b830-9c144ff1672e service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:47.159280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8f0f0a-e397-444e-b830-9c144ff1672e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:47.166148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:47.168717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f0f0a-e397-444e-b830-9c144ff1672e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:47.171110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8f0f0a-e397-444e-b830-9c144ff1672e service glance] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:47.181227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1119/2220] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:50.953307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:50.954773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:51.017966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acab313-8312-4eea-a4c1-90699c7a0587 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:51.022353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:51.022970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:51.023258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:51.023527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] PATH_INFO: `/v3/users/b04c9f0b363a4c49b4a2d1a253691b2e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:51.024625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04c9f0b363a4c49b4a2d1a253691b2e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:51.026625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:51.056983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:51.077343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04c9f0b363a4c49b4a2d1a253691b2e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.086159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04c9f0b363a4c49b4a2d1a253691b2e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.087048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04c9f0b363a4c49b4a2d1a253691b2e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.087685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04c9f0b363a4c49b4a2d1a253691b2e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.094843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04c9f0b363a4c49b4a2d1a253691b2e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.113475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1102/2221] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:50 2022] DELETE /identity/v3/users/b04c9f0b363a4c49b4a2d1a253691b2e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:51.118080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:51.118538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:51.118755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:51.141091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:51.142478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:51.200646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1120/2222] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:51.210870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:51.212271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:51.271976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb232c3-3ea6-4c15-9f09-67af735c1c6e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:51.275567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:51.276163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:51.276440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:51.276711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:51.286076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:51.287485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:51.296733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1103/2223] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:51.562362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:51.563887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:51.579652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e600be-3da5-4dd9-baee-73f108b410ff None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:51.585855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:51.586624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:51.586906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:51.587140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] PATH_INFO: `/v3/projects/ad90199955814e0c93040aaade3bf506` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:51.591559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad90199955814e0c93040aaade3bf506)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:51.593705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:51.640711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad90199955814e0c93040aaade3bf506', 'request_id': 'req-40a73609-2cb7-4028-901a-9ad38244441a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.646730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad90199955814e0c93040aaade3bf506', 'request_id': 'req-40a73609-2cb7-4028-901a-9ad38244441a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.653966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad90199955814e0c93040aaade3bf506', 'request_id': 'req-40a73609-2cb7-4028-901a-9ad38244441a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.664644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] The token cache is being invalidate because project ad90199955814e0c93040aaade3bf506 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:51.664929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.665541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.667041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1121/2224] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:51 2022] DELETE /identity/v3/projects/ad90199955814e0c93040aaade3bf506 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:51.671505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:51.681566 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504cba76-9d79-420f-ba4e-93111226da78 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:51.742306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:51.742934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:51.743213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:51.743513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] PATH_INFO: `/v3/users/a06a1b2930cf4e3788c0414a8a9c46d5` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:51.744635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a06a1b2930cf4e3788c0414a8a9c46d5)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:51.746594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:51.796975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a06a1b2930cf4e3788c0414a8a9c46d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.798753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:51.806802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a06a1b2930cf4e3788c0414a8a9c46d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.807488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a06a1b2930cf4e3788c0414a8a9c46d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.808242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a06a1b2930cf4e3788c0414a8a9c46d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.816239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a06a1b2930cf4e3788c0414a8a9c46d5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:51.839992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1104/2225] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:51 2022] DELETE /identity/v3/users/a06a1b2930cf4e3788c0414a8a9c46d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:52.116196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:52.117635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:52.131244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a73609-2cb7-4028-901a-9ad38244441a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:52.191159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:52.191916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.192284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.192569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] PATH_INFO: `/v3/projects/4662a6c75e174389adfe542182e25580` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:52.193955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4662a6c75e174389adfe542182e25580)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:52.196304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:52.221169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:52.244224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4662a6c75e174389adfe542182e25580', 'request_id': 'req-ecf0a278-bf3b-49a0-9696-e5829be512c0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.250141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4662a6c75e174389adfe542182e25580', 'request_id': 'req-ecf0a278-bf3b-49a0-9696-e5829be512c0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.257203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4662a6c75e174389adfe542182e25580', 'request_id': 'req-ecf0a278-bf3b-49a0-9696-e5829be512c0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.267798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] The token cache is being invalidate because project 4662a6c75e174389adfe542182e25580 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:52.268111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.268722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.270384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1122/2226] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:52 2022] DELETE /identity/v3/projects/4662a6c75e174389adfe542182e25580 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:52.274470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:52.275902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:52.289554 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567f438c-f4f8-49d7-a5e9-49ac1637aa02 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:52.347436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:52.347958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.348228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.348496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] PATH_INFO: `/v3/users/7356c58b41784addaba03f0693308625` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:52.349589 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7356c58b41784addaba03f0693308625)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:52.351526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:52.397759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7356c58b41784addaba03f0693308625'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.406444 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7356c58b41784addaba03f0693308625'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.407048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7356c58b41784addaba03f0693308625'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.407658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7356c58b41784addaba03f0693308625'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.414800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-49fb1ffa-0ee9-49af-9402-d0bad1afeff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7356c58b41784addaba03f0693308625'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.431724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1105/2227] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:53:52 2022] DELETE /identity/v3/users/7356c58b41784addaba03f0693308625 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:52.702871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:52.704321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:52.718199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf0a278-bf3b-49a0-9696-e5829be512c0 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:52.777673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:52.778369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.778652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.778926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] PATH_INFO: `/v3/projects/73d57859e9b8410bb057723a9efe6fcc` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:52.780655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73d57859e9b8410bb057723a9efe6fcc)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:52.783111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:52.851054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73d57859e9b8410bb057723a9efe6fcc', 'request_id': 'req-0a0f9c5e-c294-4e03-b4cf-287c3b220349'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.857974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73d57859e9b8410bb057723a9efe6fcc', 'request_id': 'req-0a0f9c5e-c294-4e03-b4cf-287c3b220349'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.866127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73d57859e9b8410bb057723a9efe6fcc', 'request_id': 'req-0a0f9c5e-c294-4e03-b4cf-287c3b220349'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.879393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] The token cache is being invalidate because project 73d57859e9b8410bb057723a9efe6fcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:53:52.879617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.880287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0a0f9c5e-c294-4e03-b4cf-287c3b220349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:53:52.882089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1123/2228] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:53:52 2022] DELETE /identity/v3/projects/73d57859e9b8410bb057723a9efe6fcc => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 19:53:52.886861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.887148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.887424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:52.908640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:52.910155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:52.952398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:52.973137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1106/2229] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:52.981005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:52.981288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:52.981562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.002178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:53.003396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:53.011286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1124/2230] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:53.016055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.025678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cc4894-5b78-4950-acd2-e1402c78cf67 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.030899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.031490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.031765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.032034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.032671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:53:53.032977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.034650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.046250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1107/2231] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:53:53.050311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.059324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc6170-a06e-4c6c-b026-d0639b29d777 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.063932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.064507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.064781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.065048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.065660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.067569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.095870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1125/2232] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:53:53 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:53.100041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.110184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c11090-4fd8-4033-a0b1-78e3c8a326da admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.166197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.166727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.167006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.167280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.167974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.169952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.191838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1108/2233] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:53:53 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:53:53.195943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.206591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957113f-56f0-46e6-8a28-0f91cbc1d47e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.211124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.211713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.212020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.212286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.212914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.214541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.220303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:53.225532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1126/2234] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:53.229301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.239026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61e183-d086-4b11-946e-37f9cb67b03a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.243618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc88e66-c4d5-45ee-aeba-f281f8f8ead2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.244202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc88e66-c4d5-45ee-aeba-f281f8f8ead2 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.244471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc88e66-c4d5-45ee-aeba-f281f8f8ead2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.244737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc88e66-c4d5-45ee-aeba-f281f8f8ead2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.245371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc88e66-c4d5-45ee-aeba-f281f8f8ead2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.247165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc88e66-c4d5-45ee-aeba-f281f8f8ead2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.258661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1109/2235] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:53:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:53:53.262824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.273356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e81155-8880-49d2-a1e8-09f6c9066048 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.277976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.278620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.279999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.280283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] PATH_INFO: `/v3/projects/b9320188c1f743f1bdf221bfa06b6f9a/users/354753f56898466e8f1854745e658ffc/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.294861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9320188c1f743f1bdf221bfa06b6f9a, user_id=354753f56898466e8f1854745e658ffc, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.297481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.309803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1127/2236] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:53:53 2022] PUT /identity/v3/projects/b9320188c1f743f1bdf221bfa06b6f9a/users/354753f56898466e8f1854745e658ffc/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:53:53.315335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576ce5-7b7c-41e9-b9a8-a526a1f8e52e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.315836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576ce5-7b7c-41e9-b9a8-a526a1f8e52e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.315934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06576ce5-7b7c-41e9-b9a8-a526a1f8e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.335031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-06576ce5-7b7c-41e9-b9a8-a526a1f8e52e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:53.336222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-06576ce5-7b7c-41e9-b9a8-a526a1f8e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:53.396518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1110/2237] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:53.406260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:53.408074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:53.468412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50fab3b-772f-489a-8f68-dbf391873ffe admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:53.472003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:53.472606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:53.472885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:53.473219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:53.483344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:53.484761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:53.494029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1128/2238] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:54.178751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbb27c-fd2f-4868-910c-ed50d8105cf8 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:54.179035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbb27c-fd2f-4868-910c-ed50d8105cf8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:54.179307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fbb27c-fd2f-4868-910c-ed50d8105cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:54.214178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-78fbb27c-fd2f-4868-910c-ed50d8105cf8 None None] MFA Rules not processed for user `354753f56898466e8f1854745e658ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:54.215425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-78fbb27c-fd2f-4868-910c-ed50d8105cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:54.255643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:53:54.286389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1111/2239] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:53:54.296366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:53:54.297803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:54.356101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb678446-2e84-4d80-bbbf-1306d090cb75 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:53:54.360362 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:53:54.360956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:54.361232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:54.361499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:54.369583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:53:54.371034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:53:54.380206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1129/2240] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:53:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:53:56.798963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29f2f8-4e0e-40ed-83cc-4003f251901f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:53:56.799516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29f2f8-4e0e-40ed-83cc-4003f251901f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:53:56.799516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29f2f8-4e0e-40ed-83cc-4003f251901f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:53:56.819600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3a29f2f8-4e0e-40ed-83cc-4003f251901f None None] MFA Rules not processed for user `354753f56898466e8f1854745e658ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:53:56.820868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3a29f2f8-4e0e-40ed-83cc-4003f251901f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:53:56.828304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1112/2241] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:53:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:04.375770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=62,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:06.813766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=12 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:44.899876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:44.901309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:44.916673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7db182-2760-4eed-8ef6-7eb4bebfb760 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:44.921924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:44.922558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:44.922835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:44.923166 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] PATH_INFO: `/v3/users/30e37aad5ec141e49a4aea056c17f063` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:44.924324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e37aad5ec141e49a4aea056c17f063)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:44.926376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:44.976557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e37aad5ec141e49a4aea056c17f063'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:44.983885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e37aad5ec141e49a4aea056c17f063'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:44.984642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e37aad5ec141e49a4aea056c17f063'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:44.985291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e37aad5ec141e49a4aea056c17f063'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:44.992261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e37aad5ec141e49a4aea056c17f063'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:45.009878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1130/2242] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 19:54:44 2022] DELETE /identity/v3/users/30e37aad5ec141e49a4aea056c17f063 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:45.014287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.014769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.014865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.033536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:45.034760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:45.092293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1113/2243] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 19:54:45.101900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.103279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:45.159498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3fe6ec-756e-4152-9c56-adafea3a9554 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.163447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e494dc8c-2ca8-4987-8939-404d7f3b0d4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.164045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494dc8c-2ca8-4987-8939-404d7f3b0d4b service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.164322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494dc8c-2ca8-4987-8939-404d7f3b0d4b service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.164592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e494dc8c-2ca8-4987-8939-404d7f3b0d4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.173448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494dc8c-2ca8-4987-8939-404d7f3b0d4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.175038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e494dc8c-2ca8-4987-8939-404d7f3b0d4b service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.184313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1131/2244] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:45.464792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.466602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:45.479820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a2b8a-5a3d-46de-8787-4d6586816ded None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.484965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.486069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.486402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.486749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] PATH_INFO: `/v3/projects/8596f8b524fd417986d83d2d4b3210d1` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.488010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8596f8b524fd417986d83d2d4b3210d1)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.490120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.541107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8596f8b524fd417986d83d2d4b3210d1', 'request_id': 'req-b5ebbd9f-756f-49eb-8561-52b54c63b307'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:45.547480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8596f8b524fd417986d83d2d4b3210d1', 'request_id': 'req-b5ebbd9f-756f-49eb-8561-52b54c63b307'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:45.555373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8596f8b524fd417986d83d2d4b3210d1', 'request_id': 'req-b5ebbd9f-756f-49eb-8561-52b54c63b307'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:45.566024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] The token cache is being invalidate because project 8596f8b524fd417986d83d2d4b3210d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 19:54:45.566322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:45.566958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5ebbd9f-756f-49eb-8561-52b54c63b307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 19:54:45.568661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1114/2245] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 19:54:45 2022] DELETE /identity/v3/projects/8596f8b524fd417986d83d2d4b3210d1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:45.574769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.575052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.575347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.596416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:45.598179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:45.658813 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1132/2246] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:45.667080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.667367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.667639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.689112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:45.690370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:45.698867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1115/2247] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:54:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:45.703004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.713140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc26c8-b56e-4e9c-8517-e17b77a9369b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.717942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.718539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.718820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.719113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.719755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 19:54:45.720055 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.721631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.733039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1133/2248] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 19:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 19:54:45.737087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.746681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c795bce-bb7f-4d41-8fc8-50dfc6729251 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.751260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.751846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.752123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.752393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.753018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.755464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.782702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1116/2249] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:54:45 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:54:45.786942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.792052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:45.798849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea6b95-965b-47a9-a25a-fac82913abd4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.858265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.858805 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.859081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.859350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.859981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.861932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.884548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1134/2250] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:54:45 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:54:45.889056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.899887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d79edc-7618-4210-a2fc-af80b2b3d109 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.904462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.905044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.905318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.905583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.906255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.907822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.917677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1117/2251] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:54:45.921341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.931101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b14ab-0fb8-4959-892a-c54e736518fe admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.935944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88749663-28da-443f-8ce1-489e0e371e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.936536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749663-28da-443f-8ce1-489e0e371e03 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.936815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749663-28da-443f-8ce1-489e0e371e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.937084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88749663-28da-443f-8ce1-489e0e371e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.937720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749663-28da-443f-8ce1-489e0e371e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.939391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88749663-28da-443f-8ce1-489e0e371e03 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:45.950650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1135/2252] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:54:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 19:54:45.954564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:45.964908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fd46ae-5c74-41d2-ae5e-171342ee9e93 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:45.969549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:45.970344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:45.970667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:45.970940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] PATH_INFO: `/v3/projects/da91a2ee43f6462a8ed96e2a8291d24d/users/a7cffb2d408743e7a562f12c3c9b3692/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:45.985724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=da91a2ee43f6462a8ed96e2a8291d24d, user_id=a7cffb2d408743e7a562f12c3c9b3692, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:45.988861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:46.001472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1118/2253] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:54:45 2022] PUT /identity/v3/projects/da91a2ee43f6462a8ed96e2a8291d24d/users/a7cffb2d408743e7a562f12c3c9b3692/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:46.007161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b522b-baca-4b84-a432-59689c566201 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:46.007453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b522b-baca-4b84-a432-59689c566201 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:46.007743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b522b-baca-4b84-a432-59689c566201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:46.029631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-6a5b522b-baca-4b84-a432-59689c566201 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:46.030854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-6a5b522b-baca-4b84-a432-59689c566201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:46.095223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1136/2254] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:46.105056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:46.106899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:46.130986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:46.175912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe6dda1-f574-45c0-9ead-3f037c498c4d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:46.179625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:46.180223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:46.180498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:46.180765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:46.189983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:46.192015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:46.201230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1119/2255] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:54:50.304296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.305185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.305546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.337021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] MFA Rules not processed for user `a7cffb2d408743e7a562f12c3c9b3692`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:50.338264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:50.412997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1137/2256] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:54:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:50.414366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:50.436516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f3f1fe-b367-4487-b8fd-74d7233ad65c service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:50.444885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:50.449972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.450517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.450817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.451570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:50.453618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:50.501799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1120/2257] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 19:54:50 2022] POST /identity/v3/projects => generated 377 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 19:54:50.506875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:50.521551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3607f92-6711-4a4a-a289-3804b426ad18 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:50.531446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:50.615684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:50.616304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.616584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.617031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.617905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:50.619861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:50.649764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1138/2258] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 19:54:50 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 19:54:50.651831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:50.663608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b460ad-064a-4e4b-aa2b-5fa39a403c28 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:50.668630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:50.669429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.669707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.670072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.670812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:50.672476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:50.685188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1121/2259] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:54:50.688955 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:50.700733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabe7674-19b8-4fbf-bc98-4b73b28a00fd admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:50.705653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:50.706364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.706684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.706956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] PATH_INFO: `/v3/projects/ad554853e2d64cc5beb5ebd562d8d3f8/users/087dcdc6aca24454a47d09d0b2da9b36/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.724043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad554853e2d64cc5beb5ebd562d8d3f8, user_id=087dcdc6aca24454a47d09d0b2da9b36, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:50.726748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:50.740801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1139/2260] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:54:50 2022] PUT /identity/v3/projects/ad554853e2d64cc5beb5ebd562d8d3f8/users/087dcdc6aca24454a47d09d0b2da9b36/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:50.744803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:50.755918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e08115-e0c6-42d3-bf30-0951173339c5 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:50.817049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852fe153-4ae8-421d-87f1-6a0e52d1df2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:50.817648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe153-4ae8-421d-87f1-6a0e52d1df2b admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.817944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe153-4ae8-421d-87f1-6a0e52d1df2b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.818211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852fe153-4ae8-421d-87f1-6a0e52d1df2b admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.818857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fe153-4ae8-421d-87f1-6a0e52d1df2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:50.820428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852fe153-4ae8-421d-87f1-6a0e52d1df2b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:50.837253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1122/2261] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 19:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 19:54:50.841331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:50.853372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50e2abe-fc58-4a51-8082-7d01dd30976b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:50.859245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:50.859985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:50.860269 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:50.860546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] PATH_INFO: `/v3/projects/ad554853e2d64cc5beb5ebd562d8d3f8/users/087dcdc6aca24454a47d09d0b2da9b36/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:50.863289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad554853e2d64cc5beb5ebd562d8d3f8, user_id=087dcdc6aca24454a47d09d0b2da9b36, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:50.865985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:50.877367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1140/2262] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 19:54:50 2022] PUT /identity/v3/projects/ad554853e2d64cc5beb5ebd562d8d3f8/users/087dcdc6aca24454a47d09d0b2da9b36/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 19:54:55.405856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:55.406389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:55.406389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:55.435423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] MFA Rules not processed for user `087dcdc6aca24454a47d09d0b2da9b36`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:54:55.436573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:55.525855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1123/2263] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:54:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:54:55.540354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:54:55.542232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:54:55.629930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4900-421c-47c7-9b9a-8976897ed7d6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:54:55.636214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:54:55.636851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:54:55.637134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:54:55.637412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:54:55.702852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:54:55.705928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:54:55.712495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:54:55.729786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1141/2264] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:54:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:05.510585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:35.129408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:35.131384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:35.188650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d849f5-5830-4076-9ab4-b64542360635 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:35.192588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:35.193671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:35.193970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:35.194235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:35.195963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:35.253132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:35.254564 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180e4114-e568-4d55-b299-54f947c07750 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:35.263817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1124/2265] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:42.513893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:42.577903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c9466c-7d23-4b55-b2dd-e80a8cad6b0c service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:42.581474 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:42.582244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:42.582595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:42.582808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:42.585172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:42.595399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:42.596909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:42.605276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1142/2266] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:45.104409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:45.104841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:45.104988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:45.124548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] MFA Rules not processed for user `eff856b09e5944a9bdcf7374cd9d4a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 19:55:45.125774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.132678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1125/2267] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 19:55:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 19:55:45.170481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:45.171908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.226532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008eff6d-7614-4604-9b76-f03e9f4f42b1 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:45.230076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:45.230669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:45.230969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:45.231237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:45.232878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.245673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:45.247094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:45.257564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1143/2268] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:45.536577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:45.538050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.549685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74e370-512c-47bd-8fdf-ecab24370197 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:45.553278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:45.553913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:45.554190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:45.554478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:45.556126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.565372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:45.566820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:45.575207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1126/2269] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:45.894319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:45.895873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.901621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:45.911908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8025b7c-4cc8-4ebb-94ff-6518d9b918ff service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:45.916575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:45.917196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:45.917437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:45.917706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:45.919556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:45.935745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:45.937177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:45.953766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1127/2270] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:45.982015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bba3e0-04fb-49b2-8a5f-c4c2f9f00b61 service glance] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:45.986825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:45.987425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:45.987705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:45.988675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:45.991033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:46.002894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:46.004470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:46.014994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1144/2271] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:48.782808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:48.784363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:48.799661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce7bb56-a3d5-4374-9364-0a6d144dffe9 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:48.805299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:48.805977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:48.806436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:48.806704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:48.808441 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:48.821131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:48.822844 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:48.834365 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1128/2272] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:55:56.010734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=84 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:58.830712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=69 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:55:58.924794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:55:58.936911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b6f79f-8369-411b-81dd-b804161b1a29 service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:55:58.943001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:55:58.943711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:55:58.944053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:55:58.945026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:55:58.946831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:55:58.959046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:55:58.961162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:55:58.971748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1145/2273] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:55:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:08.967292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:13.497377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:13.508596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9f9bd0-8e2e-4b81-84f5-0c73db450d9b service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:13.514095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:13.514685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:13.514961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:13.515230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:13.516938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:13.577142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:13.578741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:13.588419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1129/2274] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:23.583552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=28 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:56:28.292945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:56:28.304551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce7d69e-2768-4714-9f5d-058882207638 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:56:28.310209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:56:28.310811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:56:28.311093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:56:28.311366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:56:28.313688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:56:28.372387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:56:28.373980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:56:28.382896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1146/2275] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:56:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:56:38.379305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=28 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:57:29.558349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:57:29.559749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:29.571408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa5a28-f364-43e8-adf7-df4a757b5027 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:57:29.574918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:57:29.575502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:57:29.575778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:57:29.576046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:57:29.577673 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:57:29.635660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:57:29.637055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:57:29.646573 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1130/2276] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:57:29 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:57:39.641747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=28 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:58:53.982118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:58:54.019864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a20008-796f-4628-ba3c-ac31e42e30f0 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:58:54.023800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:58:54.024389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:58:54.024668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:58:54.024942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:58:54.026701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:58:54.108822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:58:54.110227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:58:54.119332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1147/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:58:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:04.115483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=50 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:33.418912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:33.469767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60caa5fb-b073-4670-830c-275d19bec8bd service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:33.474883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:33.475476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:33.475755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:33.476025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:33.477755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 19:59:33.501693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:33.503275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:33.513544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1131/2278] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 19:59:43.509403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=29 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 19:59:56.066539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 19:59:56.075035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45d759-a70b-487f-b67e-c4cb8bb34214 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 19:59:56.078583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 19:59:56.079184 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 19:59:56.079440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 19:59:56.079706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 19:59:56.088326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 19:59:56.089787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 19:59:56.098216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1148/2279] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 19:59:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:06.094383 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:34.920215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:34.948638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb121c9-31bd-4be3-aa99-47596f1ca80e service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:34.954344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:34.956247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:34.956247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:34.956247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:34.963930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:34.965494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:34.992272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1132/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:34 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:00:44.989626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=20 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:00:54.408747 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:00:54.417344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4235193f-09cd-4d43-b5c4-b303dbdce199 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:00:54.421067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:00:54.421657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:00:54.421968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:00:54.422276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:00:54.432295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:00:54.433641 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:00:54.445241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1149/2281] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:00:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:04.440364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:13.285384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:13.294672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc3645-3c11-4a89-9c60-9020b4258d00 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:13.298964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:13.299653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:13.299930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:13.300201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:13.308570 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:13.310011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:13.336720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1133/2282] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:23.334152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=13 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:29.073753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:29.092968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade0c6d-7f84-4f4c-89c3-423ea13309cb service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:29.096658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:29.097246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:29.097522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:29.097821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:29.105377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:29.106822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:29.115137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1150/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:29.121395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:29.130553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294a724-cd7e-4a9f-9c3f-bc6e2782246b service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:29.134117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:29.134722 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:29.134978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:29.135243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:29.143588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:29.144991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:29.172819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1134/2284] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:37.941681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:37.950247 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0335be-099a-409a-a951-ffac6c17e378 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:37.954506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33c7ea1-6217-4bdb-b65d-40ac93a05be6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:37.955120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c7ea1-6217-4bdb-b65d-40ac93a05be6 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:37.955396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c7ea1-6217-4bdb-b65d-40ac93a05be6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:37.955662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33c7ea1-6217-4bdb-b65d-40ac93a05be6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:37.965619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c7ea1-6217-4bdb-b65d-40ac93a05be6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:37.967047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33c7ea1-6217-4bdb-b65d-40ac93a05be6 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:37.978995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1151/2285] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:39.169798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=13 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:40.307165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:40.321623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce5c723-366b-4e9f-bce2-152784ba1649 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:40.327540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:40.328642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:40.328933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:40.329436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:40.339610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:40.341443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:40.355892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1135/2286] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:43.361197 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:43.361676 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:43.361953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:43.381552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] MFA Rules not processed for user `6a68278c8556492497def971fb1674a8`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:01:43.382787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:43.389590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1152/2287] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:01:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:01:43.695952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:43.705024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fce95b-ed62-4d4e-88a5-2a2045767a63 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:43.708497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:43.709078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:43.709350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:43.709615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:43.716653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:43.718085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:43.726219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1136/2288] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 31 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:44.047521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:44.069043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa8976-c798-4d84-b49f-490cf74010cb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:44.073951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:44.074778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:44.075057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:44.075320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:44.085113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:44.086750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:44.098431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1153/2289] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:52.311969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:52.325606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c4a9f-f1e3-4c8c-b3a9-741a822070ce service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:52.331641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:52.332235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:52.332507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:52.332775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:52.334524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:01:52.360646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:52.362246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:52.372829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1137/2290] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:01:54.093566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=25 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:01:55.729846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:01:55.744942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d624a5f-7d8f-499a-a7ac-db2f181b00bf service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:01:55.762835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:01:55.762835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:01:55.762835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:01:55.762835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:01:55.771221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:01:55.772910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:01:55.784873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1154/2291] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:01:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:02:02.368242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=23 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:02:05.779776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:10.796794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:10.808160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380cc207-399c-4ebc-900c-dcdfb2d8e60f service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:10.813542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:10.814158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:10.814426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:10.814688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:10.822190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:10.824344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:10.850649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1138/2292] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:03:20.848080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=13 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:03:53.668550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:03:53.677921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4489c2b8-f035-4613-80c5-ee3877090015 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:03:53.681382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:03:53.681992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:03:53.682265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:03:53.682532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:03:53.690515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:03:53.691922 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:03:53.700460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1155/2293] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:03:53 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:03.696624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:41.385680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:41.388782 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:41.404827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470cf1ca-693b-4a9b-86c9-3fcaa8a8a3be service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:41.409944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:41.410631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:41.411084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:41.411424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:41.422550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:41.422929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:41.433031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1139/2294] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:04:51.429029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:04:56.289024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:04:56.291818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5865063-c184-4555-b6ea-22b0cd45f3e0 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:04:56.295440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:04:56.296029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:04:56.296310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:04:56.296582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:04:56.298415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:04:56.376868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:04:56.378325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:04:56.395993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1156/2295] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:04:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:06.393432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=41 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:05:35.090529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:05:35.100064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cd62b6-49c1-4e9d-a138-8b89bb3a4c1b service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:05:35.103641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:05:35.104232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:05:35.104509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:05:35.104779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:05:35.115676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:05:35.117124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:05:35.126280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1140/2296] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:05:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:05:45.121956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:06:13.641115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:13.693924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52610dd0-604f-4d05-89bb-8024fbef7f3d service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:13.697532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:13.698167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:13.698447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:13.698718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:13.700549 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:13.767753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:13.769091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:13.778177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1157/2297] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:06:23.774043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=53 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:06:29.924690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:29.926733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:29.997382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b56d4d6-ddbb-46ad-ae06-7c47186ea2fa service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:30.002760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:30.003349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:30.004227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:30.004496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:30.006289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:30.073587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:30.075209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:30.084121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1141/2298] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:06:37.052491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:37.054134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:37.120855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67523ba-1252-4c11-893e-9bb51a7f45e8 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:37.124334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:37.124964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:37.125239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:37.125508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:37.127339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:37.137052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:37.138511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:37.147052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1158/2299] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:06:40.079833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=61 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:06:47.142899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=31 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:06:55.113526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:55.115016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:55.127007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed27ceb-52a2-44be-8e65-96459b2c2c60 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:55.131248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:55.131849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:55.132126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:55.132395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:55.134270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:55.148332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:55.149762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:55.160478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1142/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:06:57.407161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:57.416430 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d902928-22bb-4388-9c4d-c2de2bc8e55b service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:57.420066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:57.420663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:57.420935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:57.421198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:57.423027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:57.510609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:57.513607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:06:57.514105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:57.525247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e056b2-2467-47a7-b059-be9aeabbda87 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:06:57.529251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:06:57.530104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:06:57.530541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:06:57.531172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:06:57.534212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:06:57.536179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1159/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:06:57.548960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:06:57.550454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:06:57.561195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1143/2302] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:06:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:07:00.866403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:00.937084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a1c978-db12-443a-9896-a93f16bc3189 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:00.943008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:00.943682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:00.944025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:00.944396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:00.946250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:00.958371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:00.959939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:00.970819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1160/2303] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:07:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:07:04.305878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:04.307605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:04.325249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c0b780-97e2-430b-b30e-e612bf228699 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:04.330168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:04.330795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:04.331580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:04.331847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] PATH_INFO: `/v3/users/a7cffb2d408743e7a562f12c3c9b3692` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:04.332958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7cffb2d408743e7a562f12c3c9b3692)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:04.334895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:04.380486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7cffb2d408743e7a562f12c3c9b3692'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.387891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7cffb2d408743e7a562f12c3c9b3692'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.388508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7cffb2d408743e7a562f12c3c9b3692'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.389117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7cffb2d408743e7a562f12c3c9b3692'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.398950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7cffb2d408743e7a562f12c3c9b3692'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.415952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1144/2304] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:07:04 2022] DELETE /identity/v3/users/a7cffb2d408743e7a562f12c3c9b3692 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:07:04.590918 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:04.604693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d8e90-d11b-40f4-b102-9021ecf84dde service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:04.657842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:04.664767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:04.665394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:04.665670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:04.665979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] PATH_INFO: `/v3/projects/da91a2ee43f6462a8ed96e2a8291d24d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:04.667099 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da91a2ee43f6462a8ed96e2a8291d24d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:04.669020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:04.713846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da91a2ee43f6462a8ed96e2a8291d24d', 'request_id': 'req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.719846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da91a2ee43f6462a8ed96e2a8291d24d', 'request_id': 'req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.727273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da91a2ee43f6462a8ed96e2a8291d24d', 'request_id': 'req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.737608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] The token cache is being invalidate because project da91a2ee43f6462a8ed96e2a8291d24d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:07:04.737952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.738556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.739998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1161/2305] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:07:04 2022] DELETE /identity/v3/projects/da91a2ee43f6462a8ed96e2a8291d24d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:07:04.743962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:04.745448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:04.758710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63781306-bcc6-41ad-b8da-11684210afb9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:04.813459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:04.814050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:04.814328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:04.814600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] PATH_INFO: `/v3/users/087dcdc6aca24454a47d09d0b2da9b36` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:04.828445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=087dcdc6aca24454a47d09d0b2da9b36)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:04.830604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:04.868509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:04.878684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '087dcdc6aca24454a47d09d0b2da9b36'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.885327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '087dcdc6aca24454a47d09d0b2da9b36'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.885997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '087dcdc6aca24454a47d09d0b2da9b36'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.886578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '087dcdc6aca24454a47d09d0b2da9b36'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.893375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '087dcdc6aca24454a47d09d0b2da9b36'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:04.910944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1145/2306] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:07:04 2022] DELETE /identity/v3/users/087dcdc6aca24454a47d09d0b2da9b36 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:07:05.081812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:07:05.085859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:07:05.113296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b9173-6bdd-43a8-83d8-d25df0ce9b22 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:07:05.170335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:07:05.170964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:07:05.171246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:07:05.171517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] PATH_INFO: `/v3/projects/ad554853e2d64cc5beb5ebd562d8d3f8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:07:05.182485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad554853e2d64cc5beb5ebd562d8d3f8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:07:05.184491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:07:05.229814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad554853e2d64cc5beb5ebd562d8d3f8', 'request_id': 'req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:05.236266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad554853e2d64cc5beb5ebd562d8d3f8', 'request_id': 'req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:05.244286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad554853e2d64cc5beb5ebd562d8d3f8', 'request_id': 'req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:05.255408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] The token cache is being invalidate because project ad554853e2d64cc5beb5ebd562d8d3f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:07:05.255687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:05.256409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:07:05.257946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1162/2307] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:07:05 2022] DELETE /identity/v3/projects/ad554853e2d64cc5beb5ebd562d8d3f8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:07:14.907628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=15,keystone:DELETE=5,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:07:15.251718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:10.912299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:10.913685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:10.980032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aca0e1-fa9b-4d44-b83f-e1bc6b2b90f7 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:11.028100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:11.028692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:11.028969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:11.029239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:11.030959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:11.103053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:11.104385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:11.113440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1146/2308] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:10 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:08:21.109218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=78 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:08:54.393260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:08:54.394731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:54.484820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d2ee1-65ef-43b6-ab19-f89137bf92c4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:08:54.491220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:08:54.491837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:08:54.492124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:08:54.492395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:08:54.494792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:08:54.572379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:08:54.574018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:08:54.592593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1163/2309] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:08:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:04.589977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=77 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:13.106995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:13.108540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:13.152076 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84049e5f-e3f8-4e50-9cf0-fd2aad310878 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:13.157724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:13.158355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:13.158631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:13.158904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:13.160671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:13.176142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:13.177765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:13.189646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1147/2310] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:23.184864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=14 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:09:46.356115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:09:46.357526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:46.474938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daebf41f-e56f-4567-82ba-2ff084a8a72f service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:09:46.479917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:09:46.480521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:09:46.480800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:09:46.481081 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:09:46.482866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:09:46.494332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:09:46.495876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:09:46.505714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1164/2311] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:09:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:09:56.501406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=55 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:10:34.847549 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:10:34.848938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:10:34.912336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c334903-7b50-4551-91ba-4c2e6340ea3f service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:10:34.915906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:10:34.916491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:10:34.916764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:10:34.917027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:10:34.919012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:10:34.989294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:10:34.990738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:10:35.016961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1148/2312] 10.222.0.15 () {60 vars in 1396 bytes} [Tue Sep 13 20:10:34 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:10:45.014392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=65 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:11:13.546497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:13.547964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:13.568845 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc5039d-5e18-4a26-880f-396d02bf683c service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:13.572545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:13.573141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:13.573417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:13.573685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:13.576011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:13.646581 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:13.648037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:13.667442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1165/2313] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:13 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:23.664563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=40 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:11:29.347819 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:29.357797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd425b07-bfe5-4428-a1f2-f58d77c7de6c service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:29.361340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:29.361952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:29.362243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:29.362490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:29.364159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:29.430548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:29.431951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:29.449528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1149/2314] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:39.447068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=36 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:11:54.552482 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:54.571270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc6c6cb-dfd3-4522-a335-325ae9aa5040 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:54.575561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:54.576165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:54.576445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:54.577195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:54.578656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:54.611499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:54.612913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:54.632596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1166/2315] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:11:57.608286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:11:57.617461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dfa02a-0b2b-413b-be32-ca2084524123 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:11:57.620978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:11:57.621565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:11:57.621878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:11:57.622152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:11:57.623869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:11:57.637340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:11:57.638784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:11:57.649423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1150/2316] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:11:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:12:04.630051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=22 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:12:07.644763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:12:44.062903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:12:44.064361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:12:44.075625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dbe71f-f119-4e96-800e-b6777e6940e0 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:12:44.079406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:12:44.080007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:12:44.080277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:12:44.080546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:12:44.082515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:12:44.092817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:12:44.094192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:12:44.102428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1167/2317] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:12:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:12:54.098679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:13:11.172168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:11.173588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:13:11.184712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66926d99-9d3b-4d34-a0c4-0078659b0b1c service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:11.189015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:11.189601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:11.189899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:11.190172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:11.198306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:11.199697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:11.227120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1151/2318] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:13:21.224534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=13 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:13:54.629297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:13:54.638367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c8a6ff-86cc-4e9d-9b9f-344b569e9c37 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:13:54.642039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:13:54.642627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:13:54.642905 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:13:54.643173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:13:54.651248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:13:54.652660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:13:54.662291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1168/2319] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:13:54 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:04.658282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:20.632939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:20.645952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde115ca-da42-409f-9357-169137a75927 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:20.652132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:20.652998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:20.653467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:20.653949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:20.667850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:20.670083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:20.683357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1152/2320] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:25.037346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:25.047437 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045dd202-8292-4b64-85d7-9dc693c74227 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:25.050958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:25.051541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:25.051815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:25.052085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:25.053800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:25.117216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:25.119065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:25.129689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1169/2321] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:28.200373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:28.212303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d68460-b13a-4049-bf15-1500b5744ee6 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:28.217264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:28.218123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:28.218401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:28.218670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:28.220936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:28.232474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:28.234308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:28.246891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1153/2322] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:28.276115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:28.278070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:28.300941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f2a90-2c08-4704-ba1e-635a58feb65b service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:28.306969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:28.307542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:28.307861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:28.308078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] PATH_INFO: `/v3/users/53fa55dda30e4aaa86ae599e4df2fed3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:28.309201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53fa55dda30e4aaa86ae599e4df2fed3)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:28.311189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:28.357368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53fa55dda30e4aaa86ae599e4df2fed3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.365748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53fa55dda30e4aaa86ae599e4df2fed3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.366363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53fa55dda30e4aaa86ae599e4df2fed3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.366951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53fa55dda30e4aaa86ae599e4df2fed3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.373829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b044963c-7c6b-40f3-8d3c-7d0ac00352b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53fa55dda30e4aaa86ae599e4df2fed3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.392289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1170/2323] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:14:28 2022] DELETE /identity/v3/users/53fa55dda30e4aaa86ae599e4df2fed3 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:28.614280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:28.633718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c96654-9fca-4f69-b5e8-6767803531f4 service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:28.701102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:28.701773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:28.702043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:28.702468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] PATH_INFO: `/v3/projects/f82690ff3ee94e708caf1d4f7d1979c2` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:28.704127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f82690ff3ee94e708caf1d4f7d1979c2)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:28.706075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:28.766187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f82690ff3ee94e708caf1d4f7d1979c2', 'request_id': 'req-1c0f81d3-5221-4236-bbf3-07ebd76728b2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.772254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f82690ff3ee94e708caf1d4f7d1979c2', 'request_id': 'req-1c0f81d3-5221-4236-bbf3-07ebd76728b2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.781175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f82690ff3ee94e708caf1d4f7d1979c2', 'request_id': 'req-1c0f81d3-5221-4236-bbf3-07ebd76728b2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.793258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] The token cache is being invalidate because project f82690ff3ee94e708caf1d4f7d1979c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:14:28.793548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.794212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1c0f81d3-5221-4236-bbf3-07ebd76728b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:28.795680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1154/2324] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:14:28 2022] DELETE /identity/v3/projects/f82690ff3ee94e708caf1d4f7d1979c2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:28.803415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:28.803827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:28.804232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:28.867927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:28.869159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:28.898949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:28.928390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1171/2325] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:28.936328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:28.936619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:28.936895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:28.957746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:28.958970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:28.966808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1155/2326] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:28.970925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:28.980510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089dd27a-11a3-4644-bc10-df0f224dbebc None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:28.985089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:28.985674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:28.985982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:28.986252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:28.986886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:14:28.987214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:28.988780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.000812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1172/2327] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:14:29.004835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:29.013869 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc8e090-1ea4-4fbe-8dd8-f764d608663a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:29.019137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:29.019718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.019992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.020264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.020885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:29.022815 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.058280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1156/2328] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:14:29 2022] POST /identity/v3/projects => generated 405 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:14:29.062355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:29.072670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb50d71-de4d-4540-bbe5-d01406204e4a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:29.127585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:29.128224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.128502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.128786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.129410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:29.131341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.153483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1173/2329] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:14:29 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:14:29.157450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:29.168417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44ab924-3a8c-421b-95fe-d8f559a6aec0 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:29.173006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:29.173597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.173903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.174169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.174804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:29.176385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.186024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1157/2330] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:29.190210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:29.199838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3390725c-0604-47ee-a2b6-73f166d1678c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:29.205011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2fa5b3-438d-4f05-8053-13f6842e8747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:29.205583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa5b3-438d-4f05-8053-13f6842e8747 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.205886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa5b3-438d-4f05-8053-13f6842e8747 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.206153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2fa5b3-438d-4f05-8053-13f6842e8747 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.206780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fa5b3-438d-4f05-8053-13f6842e8747 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:29.208367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2fa5b3-438d-4f05-8053-13f6842e8747 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.218569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1174/2331] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:29.222561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:29.232201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf5622-21ce-41a5-8ade-c0e5665eb65f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:29.237522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:29.238198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.238476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.238764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] PATH_INFO: `/v3/projects/6f67007f514f42efaa05268a9910abb3/users/272596c1fbce41e9be635bfa6bed5211/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.253399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:29.256839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f67007f514f42efaa05268a9910abb3, user_id=272596c1fbce41e9be635bfa6bed5211, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:29.260263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.273284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1158/2332] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:14:29 2022] PUT /identity/v3/projects/6f67007f514f42efaa05268a9910abb3/users/272596c1fbce41e9be635bfa6bed5211/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:29.278441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff126d0-4140-4008-818c-ec0a0cc9ecb3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.278933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff126d0-4140-4008-818c-ec0a0cc9ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.279041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff126d0-4140-4008-818c-ec0a0cc9ecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.298097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7ff126d0-4140-4008-818c-ec0a0cc9ecb3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:29.299301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7ff126d0-4140-4008-818c-ec0a0cc9ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:29.356891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1175/2333] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:29.367123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:29.368897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:29.424987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a779d35-67bb-44bd-a6be-45dfea04189e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:29.428515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:29.429106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:29.429380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:29.429646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:29.439042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:29.440452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:29.449655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1159/2334] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:30.449828 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea467f-7d64-4942-8353-e35617495946 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.450387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea467f-7d64-4942-8353-e35617495946 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.451528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aea467f-7d64-4942-8353-e35617495946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.485267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-9aea467f-7d64-4942-8353-e35617495946 None None] MFA Rules not processed for user `272596c1fbce41e9be635bfa6bed5211`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:30.486528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9aea467f-7d64-4942-8353-e35617495946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:30.524855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:30.584084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1176/2335] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:30 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:30.588220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:30.602645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22889b92-2e19-40fc-b7c9-b1bd4cb5a696 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:30.619142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:30.619142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.619142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.619142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.620691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:30.628155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:30.671185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1160/2336] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:14:30 2022] POST /identity/v3/projects => generated 405 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:14:30.675807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea467f-7d64-4942-8353-e35617495946 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:30.686396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aea467f-7d64-4942-8353-e35617495946 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:30.746690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:30.747309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.747592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.747869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.748515 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:30.750473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:30.773412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1177/2337] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:14:30 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:14:30.777301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:30.788657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5f9c17-ed26-4f24-9bc9-bc974afe72f9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:30.793413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:30.793980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.794366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.794635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.795270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:30.796847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:30.810084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1161/2338] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:30.814717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:30.827769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d214e0e-c8a9-4c3d-90dc-7f7db3952b54 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:30.834558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:30.835321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.835762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.836123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] PATH_INFO: `/v3/projects/ea2d5ed6b2d943519489a93ecef30edc/users/9194dea996ee44e48b3c50c700621a4c/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.854053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea2d5ed6b2d943519489a93ecef30edc, user_id=9194dea996ee44e48b3c50c700621a4c, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:30.857342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:30.886463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1178/2339] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:14:30 2022] PUT /identity/v3/projects/ea2d5ed6b2d943519489a93ecef30edc/users/9194dea996ee44e48b3c50c700621a4c/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:30.890149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:30.904750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0d841-c6c1-4acd-8941-5eb134a752d8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:30.981479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503558ff-4602-455c-8509-1f34abd8924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:30.982278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503558ff-4602-455c-8509-1f34abd8924a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:30.983292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503558ff-4602-455c-8509-1f34abd8924a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:30.983292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503558ff-4602-455c-8509-1f34abd8924a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:30.984189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503558ff-4602-455c-8509-1f34abd8924a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:30.986499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503558ff-4602-455c-8509-1f34abd8924a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:31.000485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1162/2340] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:31.002555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:31.014823 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f344080-b7e3-4018-af1c-01766176af56 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:31.019734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:31.020399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:31.020672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:31.020941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] PATH_INFO: `/v3/projects/ea2d5ed6b2d943519489a93ecef30edc/users/9194dea996ee44e48b3c50c700621a4c/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:31.023039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea2d5ed6b2d943519489a93ecef30edc, user_id=9194dea996ee44e48b3c50c700621a4c, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:31.025858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:31.038508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1179/2341] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:14:31 2022] PUT /identity/v3/projects/ea2d5ed6b2d943519489a93ecef30edc/users/9194dea996ee44e48b3c50c700621a4c/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:31.755228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:31.756190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:31.756470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:31.787741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] MFA Rules not processed for user `9194dea996ee44e48b3c50c700621a4c`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:31.789432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:31.850612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:31.877012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1163/2342] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:31 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:33.114163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:33.115920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:33.173381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:33.197140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a7efd1-1796-4592-802e-c00d3bec9340 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:33.202232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364384c3-625a-4776-b816-4ada1250427f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:33.202917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364384c3-625a-4776-b816-4ada1250427f service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:33.203131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364384c3-625a-4776-b816-4ada1250427f service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:33.203439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364384c3-625a-4776-b816-4ada1250427f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:33.206405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-364384c3-625a-4776-b816-4ada1250427f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:33.217912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364384c3-625a-4776-b816-4ada1250427f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:33.219407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364384c3-625a-4776-b816-4ada1250427f service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:33.229603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1180/2343] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:14:33.237995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04575312-ad25-45a1-bf23-38caf6ed7bd7 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:33.242701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:33.243397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:33.243595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:33.243870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] PATH_INFO: `/v3/users/272596c1fbce41e9be635bfa6bed5211` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:33.244991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] RBAC: Authorizing `identity:delete_user(user_id=272596c1fbce41e9be635bfa6bed5211)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:33.247304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:33.293037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272596c1fbce41e9be635bfa6bed5211'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.299978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272596c1fbce41e9be635bfa6bed5211'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.300649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272596c1fbce41e9be635bfa6bed5211'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.301254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272596c1fbce41e9be635bfa6bed5211'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.308487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272596c1fbce41e9be635bfa6bed5211'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.326537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1164/2344] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:14:33 2022] DELETE /identity/v3/users/272596c1fbce41e9be635bfa6bed5211 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:33.498901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364384c3-625a-4776-b816-4ada1250427f service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:33.510356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364384c3-625a-4776-b816-4ada1250427f service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:33.521453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:33.572573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:33.573205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:33.573483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:33.573802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] PATH_INFO: `/v3/projects/6f67007f514f42efaa05268a9910abb3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:33.574924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f67007f514f42efaa05268a9910abb3)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:33.576829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:33.623949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f67007f514f42efaa05268a9910abb3', 'request_id': 'req-ea2dae32-9096-4ee0-96dd-26532bf4115d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.630874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f67007f514f42efaa05268a9910abb3', 'request_id': 'req-ea2dae32-9096-4ee0-96dd-26532bf4115d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.638811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f67007f514f42efaa05268a9910abb3', 'request_id': 'req-ea2dae32-9096-4ee0-96dd-26532bf4115d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.650760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] The token cache is being invalidate because project 6f67007f514f42efaa05268a9910abb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:14:33.651046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.651776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.653298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1181/2345] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:14:33 2022] DELETE /identity/v3/projects/6f67007f514f42efaa05268a9910abb3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:33.657149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:33.659074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:33.672060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698df3c-8ccb-4951-b5b0-10c3c676618b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:33.738347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:33.738998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:33.739277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:33.739553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] PATH_INFO: `/v3/users/9194dea996ee44e48b3c50c700621a4c` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:33.740708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9194dea996ee44e48b3c50c700621a4c)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:33.742714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:33.744846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:33.788856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9194dea996ee44e48b3c50c700621a4c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.796036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9194dea996ee44e48b3c50c700621a4c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.796687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9194dea996ee44e48b3c50c700621a4c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.797352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9194dea996ee44e48b3c50c700621a4c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.805429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9602f91c-2136-49a3-bcee-1a8ec5da70dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9194dea996ee44e48b3c50c700621a4c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:33.823453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1165/2346] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:14:33 2022] DELETE /identity/v3/users/9194dea996ee44e48b3c50c700621a4c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:33.988867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:33.990595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:34.009155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2dae32-9096-4ee0-96dd-26532bf4115d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.066498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.067130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.067551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.067829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] PATH_INFO: `/v3/projects/ea2d5ed6b2d943519489a93ecef30edc` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.069066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea2d5ed6b2d943519489a93ecef30edc)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.071004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.114983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:34.118210 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea2d5ed6b2d943519489a93ecef30edc', 'request_id': 'req-8e61dc57-9486-48f3-8315-721dab4a926e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:34.125637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea2d5ed6b2d943519489a93ecef30edc', 'request_id': 'req-8e61dc57-9486-48f3-8315-721dab4a926e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:34.132596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea2d5ed6b2d943519489a93ecef30edc', 'request_id': 'req-8e61dc57-9486-48f3-8315-721dab4a926e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:34.143133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] The token cache is being invalidate because project ea2d5ed6b2d943519489a93ecef30edc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:14:34.143418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:34.144031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-8e61dc57-9486-48f3-8315-721dab4a926e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:14:34.145468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1182/2347] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:14:33 2022] DELETE /identity/v3/projects/ea2d5ed6b2d943519489a93ecef30edc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:34.150353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.150647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.150920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.170418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:34.171601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:34.227991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1166/2348] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:34.235298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.235584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.235993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.255520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:34.256638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:34.264332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1183/2349] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:34.268324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.277967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf900fb7-912f-4f40-9d45-ddfa77024021 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.282514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.283617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.283893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.284167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.284806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:14:34.285106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.286745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.298020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1167/2350] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:14:34.302576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.311594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37d930-e980-4f81-b609-4c37049d3e42 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.316309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.316889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.317161 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.317427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.318082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.319949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.346570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1184/2351] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:14:34.350699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.360562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e9cea9-c2c1-433f-ad58-bcce0585f9bd admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.416369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.417036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.417282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.417487 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.418156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.420041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.438947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:34.447381 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1168/2352] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:14:34.451472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.462420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8742b8-22dc-4910-aa18-fa47a7f5603d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.467078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.467775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.468117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.468456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.469168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.471526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.481584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1185/2353] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:34.485536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.496408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7af122-4e0f-4ed9-8ac4-3bdc7ebbd078 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.501385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0232b6-ce1a-41dd-b9e5-573aa8cb36dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.502013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0232b6-ce1a-41dd-b9e5-573aa8cb36dc admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.502417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0232b6-ce1a-41dd-b9e5-573aa8cb36dc admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.502670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0232b6-ce1a-41dd-b9e5-573aa8cb36dc admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.510229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0232b6-ce1a-41dd-b9e5-573aa8cb36dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.510229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0232b6-ce1a-41dd-b9e5-573aa8cb36dc admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.515786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1169/2354] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:34.520306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.530219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b066b7-0385-4491-8ed4-19e19753095a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.535428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.536065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.536343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.536617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] PATH_INFO: `/v3/projects/22fda80074894ca6a58dee9bc0a46205/users/49294a5bab1343b1b1d4bf8733e27a0b/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.550763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fda80074894ca6a58dee9bc0a46205, user_id=49294a5bab1343b1b1d4bf8733e27a0b, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.553392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.566146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1186/2355] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:14:34 2022] PUT /identity/v3/projects/22fda80074894ca6a58dee9bc0a46205/users/49294a5bab1343b1b1d4bf8733e27a0b/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:14:34.576005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.576291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.576569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.604689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] MFA Rules not processed for user `49294a5bab1343b1b1d4bf8733e27a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:34.605941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:34.669032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1170/2356] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:34.673547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.734679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67398b81-64d6-439d-9422-07e0e34e90fd admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.740116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.740873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.741186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.741464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.742523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.744820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.776003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1187/2357] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:14:34.777102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.788590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054d225d-9def-4564-85b9-0656b5eaea63 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.851137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.851745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.852021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.852298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.852929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.854871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.877496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1171/2358] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:14:34 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:14:34.881454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.891845 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbba3f-2324-462c-98f0-b930cf314a60 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.899076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.899683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.899956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.900221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.900861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.902496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.913412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1188/2359] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:34.917600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.927373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e591d56-8e32-4b48-8ab9-0c04bd5fe961 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:34.931926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:34.932562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:34.932836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:34.933105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] PATH_INFO: `/v3/projects/fb91ebdf5849492f89094139521756a9/users/7427d2c0bcc242f19448d68f3544cfed/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:34.947948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb91ebdf5849492f89094139521756a9, user_id=7427d2c0bcc242f19448d68f3544cfed, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:34.950799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:34.963716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1172/2360] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:14:34 2022] PUT /identity/v3/projects/fb91ebdf5849492f89094139521756a9/users/7427d2c0bcc242f19448d68f3544cfed/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:14:34.967733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:34.977519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e791bb6d-9525-4533-9611-85331a464cb0 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:35.018006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:35.034932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3513d5a8-3f7a-4208-9e04-35175065aa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:35.035461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d5a8-3f7a-4208-9e04-35175065aa33 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:35.035739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d5a8-3f7a-4208-9e04-35175065aa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:35.036008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3513d5a8-3f7a-4208-9e04-35175065aa33 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:35.036643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513d5a8-3f7a-4208-9e04-35175065aa33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:35.038256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3513d5a8-3f7a-4208-9e04-35175065aa33 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:35.047943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1189/2361] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:14:35.052125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:35.062094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf707e89-e71b-4687-8afb-b2aeaaf77971 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:35.067104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:35.067733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:35.068007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:35.068299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] PATH_INFO: `/v3/projects/fb91ebdf5849492f89094139521756a9/users/7427d2c0bcc242f19448d68f3544cfed/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:35.070263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb91ebdf5849492f89094139521756a9, user_id=7427d2c0bcc242f19448d68f3544cfed, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:35.072848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:35.083340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1173/2362] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:14:35 2022] PUT /identity/v3/projects/fb91ebdf5849492f89094139521756a9/users/7427d2c0bcc242f19448d68f3544cfed/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:14:35.094039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:35.094321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:35.094592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:35.122570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] MFA Rules not processed for user `7427d2c0bcc242f19448d68f3544cfed`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:14:35.123774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:35.191954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1190/2363] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:14:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:14:45.079127 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:45.188582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=43 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:14:51.313550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:14:51.315729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:51.382518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7683fb-69c3-4891-8e74-dc5f90ef331d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:14:51.387413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:14:51.387994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:14:51.388265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:14:51.388529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:14:51.390222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:14:51.402204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:14:51.403759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a192b6-efc9-401e-a78d-f0015d7fac9b service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:14:51.413957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1174/2364] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:14:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:01.409638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=32 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:36.048543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:36.050907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:36.134320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfcd5bc-1585-42d3-ad8b-72bf9a646ac8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:36.140226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:36.140846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:36.141127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:36.141399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:36.143250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:36.207738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:36.209657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:36.220154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1191/2365] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:38.441533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:38.442134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:38.442134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:38.469081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] MFA Rules not processed for user `eff856b09e5944a9bdcf7374cd9d4a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:15:38.470314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:38.477572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1175/2366] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:15:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:15:38.805130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:38.806566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:38.863438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a855f-d0e7-45e9-860b-deba0c5bd71f service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:38.867090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:38.867687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:38.867966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:38.868238 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:38.869978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:38.879321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:38.880737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:38.888704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1192/2367] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:40.218732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:40.220282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:40.235541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de258f9-03b8-4d2a-870b-ec28580746e8 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:40.241080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:40.241745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:40.242031 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:40.242302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:40.244005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:40.255841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:40.257505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:40.267680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1176/2368] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:48.394145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:48.395770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:48.412661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05824272-cd95-4869-8f05-77adbcfb2344 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:48.418384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:48.418958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:48.419214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:48.419487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:48.421253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:48.433759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:48.435344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:48.447303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1193/2369] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:50.263397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=12 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:15:52.105072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:15:52.117015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae879f79-fdf1-4a27-bbcd-90c6550d2fb4 service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:15:52.122376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:15:52.122964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:15:52.123242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:15:52.123508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:15:52.125187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:15:52.138187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:15:52.139843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:15:52.151177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1177/2370] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:15:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:15:58.442840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=88 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:16:02.146689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:16:16.113405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:16:16.114821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:16:16.168364 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde6e15c-8aac-4005-82e5-69b3c047e396 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:16:16.171881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:16:16.172469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:16:16.172743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:16:16.173011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:16:16.182792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:16:16.184196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:16:16.193600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1194/2371] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:16:16 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:16:26.189320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=26 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:16:29.521336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:16:29.532389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eb340-b25f-471d-94b3-c8dace40df1a service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:16:29.538469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:16:29.539051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:16:29.539323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:16:29.539672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:16:29.541350 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:16:29.597884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:16:29.599448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:16:29.608571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1178/2372] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:16:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:16:39.604158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=28 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:16:55.041086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:16:55.050609 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b472c33-e24e-43cc-bf45-340965809bfb service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:16:55.054338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:16:55.054932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:16:55.055208 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:16:55.055481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:16:55.057203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:16:55.071709 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:16:55.073107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:16:55.084513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1195/2373] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:16:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:16:58.401609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:16:58.411282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216bc8-dc47-4469-a98b-e0b63f01d882 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:16:58.414861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:16:58.415440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:16:58.415714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:16:58.415980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:16:58.417635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:16:58.431126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:16:58.433227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9aaf7-5342-4f39-92de-f27c60771e36 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:16:58.452916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1179/2374] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:16:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:05.079090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:17:08.447748 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:17:49.063729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:17:49.065187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:49.087419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b1523d-df4d-4298-8e13-6a6449559e50 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:17:49.091120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:17:49.091655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:17:49.091933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:17:49.092204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:17:49.094404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:17:49.104058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:17:49.105487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:17:49.115376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1196/2375] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:17:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:17:59.110968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=11 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:12.661325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:12.661894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:12.661894 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:12.681406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] MFA Rules not processed for user `eff856b09e5944a9bdcf7374cd9d4a2a`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:12.682649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:12.689578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1180/2376] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:12.993501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:13.002881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4deb6-6e6f-48ec-ac72-be8a12f5b214 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:13.006443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:13.007027 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:13.007302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:13.007570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:13.009263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:13.019443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:13.020843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:13.028638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1197/2377] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:16.377787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:16.390727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8367bbff-6e48-452f-8df7-64a7d770b226 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:16.394277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:16.394876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:16.395149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:16.395531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:16.397260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:16.469211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:16.470651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2603d88-7c72-4e5d-a39b-04f3f63f0138 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:16.479531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1181/2378] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:16.998739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:17.089865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9013cb7e-9440-4da6-b3fb-5bf1a74ae3de service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:17.094921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:17.095563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:17.095846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:17.096120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] PATH_INFO: `/v3/users/49294a5bab1343b1b1d4bf8733e27a0b` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:17.097254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49294a5bab1343b1b1d4bf8733e27a0b)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:17.099341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:17.159143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49294a5bab1343b1b1d4bf8733e27a0b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.166292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49294a5bab1343b1b1d4bf8733e27a0b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.166911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49294a5bab1343b1b1d4bf8733e27a0b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.167509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49294a5bab1343b1b1d4bf8733e27a0b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.175604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49294a5bab1343b1b1d4bf8733e27a0b'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.195037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1198/2379] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:18:16 2022] DELETE /identity/v3/users/49294a5bab1343b1b1d4bf8733e27a0b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:17.199518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:17.199801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:17.200107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:17.240535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:17.241841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:17.303525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1182/2380] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:17.313592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:17.315076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:17.371422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:17.375893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b37b1-daef-49db-801d-53df43636e44 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:17.380799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:17.381605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:17.382158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:17.382649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:17.391979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:17.393395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:17.402692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1199/2381] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:17.674456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:17.675921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:17.691458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a980294-3015-42d4-ac44-9231bc83c2a3 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:17.696227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:17.697020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:17.697248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:17.697516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] PATH_INFO: `/v3/projects/22fda80074894ca6a58dee9bc0a46205` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:17.698734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22fda80074894ca6a58dee9bc0a46205)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:17.700779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:17.747946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22fda80074894ca6a58dee9bc0a46205', 'request_id': 'req-2f95e52b-1788-435b-9daa-c1220ec619b4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.754029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22fda80074894ca6a58dee9bc0a46205', 'request_id': 'req-2f95e52b-1788-435b-9daa-c1220ec619b4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.761144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22fda80074894ca6a58dee9bc0a46205', 'request_id': 'req-2f95e52b-1788-435b-9daa-c1220ec619b4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.765028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:17.773718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] The token cache is being invalidate because project 22fda80074894ca6a58dee9bc0a46205 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:18:17.774232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.774855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.776310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1183/2382] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:18:17 2022] DELETE /identity/v3/projects/22fda80074894ca6a58dee9bc0a46205 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:17.780387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:17.790712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a7cf56-9a87-4357-9dfb-2cc437cf89f2 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:17.847770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:17.848393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:17.848667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:17.848936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] PATH_INFO: `/v3/users/7427d2c0bcc242f19448d68f3544cfed` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:17.850075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7427d2c0bcc242f19448d68f3544cfed)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:17.851995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:17.897624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7427d2c0bcc242f19448d68f3544cfed'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.905539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7427d2c0bcc242f19448d68f3544cfed'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.907065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7427d2c0bcc242f19448d68f3544cfed'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.907670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7427d2c0bcc242f19448d68f3544cfed'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.915173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-1af7d5ab-efd7-4ef6-83b7-dfb69489f84f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7427d2c0bcc242f19448d68f3544cfed'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:17.932875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1200/2383] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:18:17 2022] DELETE /identity/v3/users/7427d2c0bcc242f19448d68f3544cfed => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:18.295045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.296524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:18.312823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95e52b-1788-435b-9daa-c1220ec619b4 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.373279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.373679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.375069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.375344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] PATH_INFO: `/v3/projects/fb91ebdf5849492f89094139521756a9` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.376498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb91ebdf5849492f89094139521756a9)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.378461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.423546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb91ebdf5849492f89094139521756a9', 'request_id': 'req-93a60e01-0856-4b44-90d0-2432ee15a8a0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:18.429508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb91ebdf5849492f89094139521756a9', 'request_id': 'req-93a60e01-0856-4b44-90d0-2432ee15a8a0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:18.436588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb91ebdf5849492f89094139521756a9', 'request_id': 'req-93a60e01-0856-4b44-90d0-2432ee15a8a0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:18.447266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] The token cache is being invalidate because project fb91ebdf5849492f89094139521756a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:18:18.447553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:18.448172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-93a60e01-0856-4b44-90d0-2432ee15a8a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:18.449658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1184/2384] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:18:18 2022] DELETE /identity/v3/projects/fb91ebdf5849492f89094139521756a9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:18.455197 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.455656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.455777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.474923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:18.476129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:18.533073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1201/2385] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:18.540806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.541087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.541359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.561312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:18.563156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:18.570860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1185/2386] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:18.574903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.584525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eadb026-9eaa-4068-ae2f-86afea81dcd2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.589011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.589597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.589910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.590179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.590808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:18:18.591103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.592686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.599243 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:18.605218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1202/2387] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:18:18.610640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.621426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6be838-ac89-4b5b-87c2-57eb3c7a435d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.626949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.627551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.627833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.628104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.628733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.630692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.658221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1186/2388] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:18:18 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:18:18.662877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.674317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f346c24d-8ba9-499e-bf2d-3c86ff3af2ac admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.729861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.730456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.730735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.731006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.731622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.733559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.756410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1203/2389] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:18:18 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:18:18.760452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.770516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9447a-2edc-446e-b3f0-b502536e1aca admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.775006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.775590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.775867 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.776138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.776774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.778387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.788339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1187/2390] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:18:18.791931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.802219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6150082-11fd-459e-b46e-d3a2cacc072e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.806880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2da1cf-ecfb-4c82-90ed-9f9d34cbf338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.807471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2da1cf-ecfb-4c82-90ed-9f9d34cbf338 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.807746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2da1cf-ecfb-4c82-90ed-9f9d34cbf338 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.808011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2da1cf-ecfb-4c82-90ed-9f9d34cbf338 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.808639 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2da1cf-ecfb-4c82-90ed-9f9d34cbf338 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.810236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2da1cf-ecfb-4c82-90ed-9f9d34cbf338 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.821815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1204/2391] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:18:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:18:18.825695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:18.835822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb238ccf-f4b8-4916-bfd3-afac262af24c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:18.840417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:18.841061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.841337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.841608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] PATH_INFO: `/v3/projects/d6abdecc0bae451cbfce2734e379eec7/users/29bb03f995034d4b8ecd3ccd214261ff/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.855966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6abdecc0bae451cbfce2734e379eec7, user_id=29bb03f995034d4b8ecd3ccd214261ff, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:18.858619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:18.872075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1188/2392] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:18:18 2022] PUT /identity/v3/projects/d6abdecc0bae451cbfce2734e379eec7/users/29bb03f995034d4b8ecd3ccd214261ff/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:18.884250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:18.884534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:18.884803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:18.911795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] MFA Rules not processed for user `29bb03f995034d4b8ecd3ccd214261ff`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:18.912995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:18.977634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1205/2393] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:18.982122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:19.003469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:19.049914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d664747e-23b8-48b2-9454-6135d9bdf9bb admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:19.054743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:19.055345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.055625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.056007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.056828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:19.058914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:19.086077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1189/2394] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:18:18 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:18:19.090069 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:19.101345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3c35d-857c-4601-92cd-0b00bc725bf0 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:19.165894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:19.166175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.166418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.166678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.167314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:19.169215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:19.182083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:19.201841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1206/2395] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:18:19 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:18:19.205346 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:19.220256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a76612-e5b8-440f-94d4-02f6f4d81e94 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:19.227084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:19.227779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.228119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.228451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.229162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:19.231208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:19.245800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1190/2396] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:18:19.247219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:19.257939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a5ea5f-7998-4803-acf4-b8e0312cb6ac admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:19.265141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:19.265860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.266142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.266414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] PATH_INFO: `/v3/projects/bd247179e076487faff733fd938ffa4c/users/09c9eca7889447bcb1b5f07f15476618/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.285498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd247179e076487faff733fd938ffa4c, user_id=09c9eca7889447bcb1b5f07f15476618, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:19.288192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:19.304966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1207/2397] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:18:19 2022] PUT /identity/v3/projects/bd247179e076487faff733fd938ffa4c/users/09c9eca7889447bcb1b5f07f15476618/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:19.308776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:19.318832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f263562-270a-4317-a41b-8416a37f14d9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:19.377451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c4ca7f-133d-451f-b98f-cf99c3f7db41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:19.378086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ca7f-133d-451f-b98f-cf99c3f7db41 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.378367 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ca7f-133d-451f-b98f-cf99c3f7db41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.378638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c4ca7f-133d-451f-b98f-cf99c3f7db41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.379277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ca7f-133d-451f-b98f-cf99c3f7db41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:19.380878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c4ca7f-133d-451f-b98f-cf99c3f7db41 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:19.391056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1191/2398] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:18:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:18:19.395293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:19.405301 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68774d7-db95-47c7-94d3-b48d94fe8d02 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:19.411698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:19.412331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.412606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.412876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] PATH_INFO: `/v3/projects/bd247179e076487faff733fd938ffa4c/users/09c9eca7889447bcb1b5f07f15476618/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.414771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd247179e076487faff733fd938ffa4c, user_id=09c9eca7889447bcb1b5f07f15476618, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:19.417411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:19.427246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1208/2399] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:18:19 2022] PUT /identity/v3/projects/bd247179e076487faff733fd938ffa4c/users/09c9eca7889447bcb1b5f07f15476618/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:18:19.437157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:19.437442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:19.437722 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:19.466955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] MFA Rules not processed for user `09c9eca7889447bcb1b5f07f15476618`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:19.468829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:19.534784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1192/2400] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:29.423681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=17,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:29.532104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:31.153718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:31.155544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:31.224848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d7e4e-7d81-4eee-a587-7808dbe25483 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:31.228372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d86faa-6f2c-4466-909d-9584ca2125a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:31.228962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86faa-6f2c-4466-909d-9584ca2125a6 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:31.229237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86faa-6f2c-4466-909d-9584ca2125a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:31.229504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d86faa-6f2c-4466-909d-9584ca2125a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:31.238935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d86faa-6f2c-4466-909d-9584ca2125a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:31.240390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d86faa-6f2c-4466-909d-9584ca2125a6 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:31.249850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1209/2401] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:31 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:31.277966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:31.340479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b45c8bd-f3a1-4ce5-82e6-f3f7fdd53c62 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:31.345585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:31.346165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:31.346437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:31.346700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] PATH_INFO: `/v3/users/29bb03f995034d4b8ecd3ccd214261ff` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:31.347801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29bb03f995034d4b8ecd3ccd214261ff)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:31.349709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:31.395128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29bb03f995034d4b8ecd3ccd214261ff'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.403622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29bb03f995034d4b8ecd3ccd214261ff'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.404387 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29bb03f995034d4b8ecd3ccd214261ff'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.404984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29bb03f995034d4b8ecd3ccd214261ff'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.412827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29bb03f995034d4b8ecd3ccd214261ff'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.429249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1193/2402] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:18:31 2022] DELETE /identity/v3/users/29bb03f995034d4b8ecd3ccd214261ff => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:31.434056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:31.434340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:31.434615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:31.453978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:31.455136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:31.514117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1210/2403] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:31.524206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:31.525599 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:31.583146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e06f1e7-1004-49fd-8086-b95a84393256 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:31.586808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:31.587403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:31.587679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:31.587949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:31.597452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:31.598842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:31.608063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1194/2404] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:18:31.870340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:31.871990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:31.885974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cd5eb-828f-4774-9bff-1f4cee322593 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:31.890593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:31.891725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:31.891998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:31.892265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] PATH_INFO: `/v3/projects/d6abdecc0bae451cbfce2734e379eec7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:31.893375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6abdecc0bae451cbfce2734e379eec7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:31.895337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:31.944594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6abdecc0bae451cbfce2734e379eec7', 'request_id': 'req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.950776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6abdecc0bae451cbfce2734e379eec7', 'request_id': 'req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.959032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6abdecc0bae451cbfce2734e379eec7', 'request_id': 'req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.969434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] The token cache is being invalidate because project d6abdecc0bae451cbfce2734e379eec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:18:31.969718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.970358 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:31.971822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1211/2405] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:18:31 2022] DELETE /identity/v3/projects/d6abdecc0bae451cbfce2734e379eec7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:31.975878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:31.985461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7baa622-5207-46dd-90d0-a0d363ed7fc4 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.003393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:32.043082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.043693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.043964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.044231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] PATH_INFO: `/v3/users/09c9eca7889447bcb1b5f07f15476618` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.045323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09c9eca7889447bcb1b5f07f15476618)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.047752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.092050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09c9eca7889447bcb1b5f07f15476618'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.098618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09c9eca7889447bcb1b5f07f15476618'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.099437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09c9eca7889447bcb1b5f07f15476618'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.100038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09c9eca7889447bcb1b5f07f15476618'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.106868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a82c852d-342d-4022-9fd5-33e76d7c42d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09c9eca7889447bcb1b5f07f15476618'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.124090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1195/2406] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:18:31 2022] DELETE /identity/v3/users/09c9eca7889447bcb1b5f07f15476618 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:32.390127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.391682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:32.405275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee16e66-6f6a-40b3-8e05-1b2c0a1c1d56 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.418043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:32.463564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.464186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.464463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.464733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] PATH_INFO: `/v3/projects/bd247179e076487faff733fd938ffa4c` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.465852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd247179e076487faff733fd938ffa4c)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.467852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.513684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd247179e076487faff733fd938ffa4c', 'request_id': 'req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.519593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd247179e076487faff733fd938ffa4c', 'request_id': 'req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.526739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd247179e076487faff733fd938ffa4c', 'request_id': 'req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.538499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] The token cache is being invalidate because project bd247179e076487faff733fd938ffa4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:18:32.538793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.539439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a43962ed-2c36-4ed0-b0cf-ec6586d09ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:18:32.540911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1212/2407] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:18:32 2022] DELETE /identity/v3/projects/bd247179e076487faff733fd938ffa4c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:32.545937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.546464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.546464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.567391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:32.568736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:32.634340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1196/2408] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:32.641377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.641752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.642192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.662687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:32.663945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:32.671699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1213/2409] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:32.676465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.686263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a44a02-3cec-4c4d-b2ad-e8073ebfeec6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.690794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.691385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.691659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.691936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.692575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:18:32.692875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.694483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.705914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1197/2410] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:18:32.709937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.719540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f60f6a-a5e0-46a9-824f-ce06d6b7eaa3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.724070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.724660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.724933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.725201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.725853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.727718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.754131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1214/2411] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:18:32 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:18:32.758102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.768481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951c400d-d67a-4eda-aa8b-a1c7336efb52 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.794713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:32.826775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.827370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.827642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.827906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.829025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.830959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.853314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1198/2412] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:18:32 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:18:32.857421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.867800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e935b688-17f4-4df5-ad4a-44d190fa575b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.872366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.872949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.873224 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.873493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.874165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.875766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.888466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1215/2413] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:18:32.893040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.903064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b55d68-465a-466c-bfcc-42bb010852e1 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.908408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2897d-c44d-4d84-b733-8ecc9b087e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.909005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2897d-c44d-4d84-b733-8ecc9b087e9a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.909279 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2897d-c44d-4d84-b733-8ecc9b087e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.909547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2897d-c44d-4d84-b733-8ecc9b087e9a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.910492 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2897d-c44d-4d84-b733-8ecc9b087e9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.912100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2897d-c44d-4d84-b733-8ecc9b087e9a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.922739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1199/2414] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:18:32.926873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:18:32.936836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaa42b-8934-4de8-925d-3198547a10cd admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:18:32.941420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:18:32.942094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.942374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.942649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] PATH_INFO: `/v3/projects/5f852236668a4f7aacd552c0d0ddc3d4/users/ce67f562dc58479492872251353edf5f/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:32.957344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f852236668a4f7aacd552c0d0ddc3d4, user_id=ce67f562dc58479492872251353edf5f, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:18:32.960050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:18:32.973255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1216/2415] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:18:32 2022] PUT /identity/v3/projects/5f852236668a4f7aacd552c0d0ddc3d4/users/ce67f562dc58479492872251353edf5f/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:18:32.984512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a979b6-a700-4e24-b6a7-8b58695d273a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:18:32.984795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a979b6-a700-4e24-b6a7-8b58695d273a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:18:32.985066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a979b6-a700-4e24-b6a7-8b58695d273a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:18:33.012475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-91a979b6-a700-4e24-b6a7-8b58695d273a None None] MFA Rules not processed for user `ce67f562dc58479492872251353edf5f`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:18:33.013674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-91a979b6-a700-4e24-b6a7-8b58695d273a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:18:33.077345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1200/2416] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:18:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:18:42.969339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:18:43.074511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=53,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:33.141065 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:33.201456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f8af5-8a36-4935-96a6-7c4d6767670f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:33.205951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:33.206566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:33.206838 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:33.207103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] PATH_INFO: `/v3/users/ce67f562dc58479492872251353edf5f` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:33.208186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce67f562dc58479492872251353edf5f)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:33.210125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:33.259294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce67f562dc58479492872251353edf5f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.267171 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce67f562dc58479492872251353edf5f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.267850 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce67f562dc58479492872251353edf5f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.268526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce67f562dc58479492872251353edf5f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.275504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce67f562dc58479492872251353edf5f'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.292897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1217/2417] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:19:33 2022] DELETE /identity/v3/users/ce67f562dc58479492872251353edf5f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:19:33.297558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:33.298041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:33.298177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:33.327541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:19:33.328809 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:33.385962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1201/2418] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:19:33.396043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:33.397389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:33.461977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3282d7-333b-4f0d-a9f4-c119861c3375 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:33.465495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50617278-4b3a-4748-8e30-e103aba6f087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:33.466121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617278-4b3a-4748-8e30-e103aba6f087 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:33.466394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617278-4b3a-4748-8e30-e103aba6f087 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:33.466661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50617278-4b3a-4748-8e30-e103aba6f087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:33.475919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50617278-4b3a-4748-8e30-e103aba6f087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:33.477320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50617278-4b3a-4748-8e30-e103aba6f087 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:33.486509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1218/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:19:33.748852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:33.750473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:33.762426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d09fb4-b8f0-4b5f-8f7a-11b6eaa6fa45 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:33.767149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:33.767995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:33.768277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:33.769084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] PATH_INFO: `/v3/projects/5f852236668a4f7aacd552c0d0ddc3d4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:33.770263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f852236668a4f7aacd552c0d0ddc3d4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:33.772183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:33.819954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f852236668a4f7aacd552c0d0ddc3d4', 'request_id': 'req-118b49de-4ca7-4482-b524-af894ab3018e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.826116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f852236668a4f7aacd552c0d0ddc3d4', 'request_id': 'req-118b49de-4ca7-4482-b524-af894ab3018e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.834601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f852236668a4f7aacd552c0d0ddc3d4', 'request_id': 'req-118b49de-4ca7-4482-b524-af894ab3018e'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.845591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] The token cache is being invalidate because project 5f852236668a4f7aacd552c0d0ddc3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:19:33.845935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.846557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-118b49de-4ca7-4482-b524-af894ab3018e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:19:33.849666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1202/2420] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:19:33 2022] DELETE /identity/v3/projects/5f852236668a4f7aacd552c0d0ddc3d4 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:19:33.858014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:33.858312 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:33.858617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:33.880061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:19:33.881295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:33.895605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:33.948691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1219/2421] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:19:33.957092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:33.957378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:33.957670 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:33.976896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:19:33.978131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:33.985936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1203/2422] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:19:33.989879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:34.000290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b89b47-3a47-4e73-b4e2-08d709ae0f98 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:34.005344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:34.005964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.006241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.006507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.007137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:19:34.007432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:34.008995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:34.020698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1220/2423] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:19:34.024686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:34.033720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26acf8-6632-4fa8-8a66-153ac024be5e None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:34.038652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:34.039234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.039507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.039775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.040576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:34.042496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:34.069544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1204/2424] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:19:34 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:19:34.073606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:34.084066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046dada9-88b9-4a85-96aa-e88f64ea99d3 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:34.138699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:34.139293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.139573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.139846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.140469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:34.142414 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:34.163507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1221/2425] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:19:34 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:19:34.168453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:34.180588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5ad59-aa2a-4e2b-8753-d76e861d0e95 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:34.185385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:34.186023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.186304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.186593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.187274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:34.188846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:34.199703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1205/2426] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:19:34.203689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:34.213646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9800fcc-5056-4ca7-becf-9efda40ecc41 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:34.218271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b9e8b7-d90c-4147-bf28-a02fb3103e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:34.218814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9e8b7-d90c-4147-bf28-a02fb3103e5f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.219093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9e8b7-d90c-4147-bf28-a02fb3103e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.219367 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b9e8b7-d90c-4147-bf28-a02fb3103e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.220006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9e8b7-d90c-4147-bf28-a02fb3103e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:34.221591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b9e8b7-d90c-4147-bf28-a02fb3103e5f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:34.232028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1222/2427] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:19:34.236069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:34.246341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c66f6f1-62ad-42c8-8e27-1fcfdb429a62 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:34.256781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:34.257436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.257709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.258010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] PATH_INFO: `/v3/projects/93254702586648759994449e9d63bfd5/users/b6deb6f1f00140e9ac92c9255b0ed71e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.272424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93254702586648759994449e9d63bfd5, user_id=b6deb6f1f00140e9ac92c9255b0ed71e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:34.275081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:34.287525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1206/2428] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:19:34 2022] PUT /identity/v3/projects/93254702586648759994449e9d63bfd5/users/b6deb6f1f00140e9ac92c9255b0ed71e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:19:34.297927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec8bd7-e368-43b7-8782-f98932b55f8e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:34.298214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec8bd7-e368-43b7-8782-f98932b55f8e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:34.298486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ec8bd7-e368-43b7-8782-f98932b55f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:34.326692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c2ec8bd7-e368-43b7-8782-f98932b55f8e None None] MFA Rules not processed for user `b6deb6f1f00140e9ac92c9255b0ed71e`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:19:34.327890 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c2ec8bd7-e368-43b7-8782-f98932b55f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:34.393091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1223/2429] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:19:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:19:44.283607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:44.390356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:19:53.306538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:19:53.308320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:53.417301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c20e62-64f6-4c51-8491-482dc1ed8f25 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:19:53.422307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:19:53.422913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:19:53.423191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:19:53.423463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:19:53.425174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:19:53.436499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:19:53.438104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:19:53.448265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1207/2430] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:19:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:20:03.444218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=50 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:20:30.856562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:20:30.857039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:20:30.857145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:20:30.896470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] MFA Rules not processed for user `91a5130420764ac78383961d7665a964`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:20:30.897691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:20:30.952038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1224/2431] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:20:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:20:31.340693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:20:31.342179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:20:31.399057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ea699-41e6-4aae-95ff-01984cf5ebac service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:20:31.402690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:20:31.403301 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:20:31.403577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:20:31.403854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:20:31.405600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:20:31.488934 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:20:31.490407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:20:31.509317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1208/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:20:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:20:31.536594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:20:31.546249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4721cc8-1489-4cc8-a30d-651eb8f7f304 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:20:31.549866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:20:31.550510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:20:31.550794 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:20:31.551064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:20:31.552796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] Could not find project: 5f852236668a4f7aacd552c0d0ddc3d4.: keystone.exception.ProjectNotFound: Could not find project: 5f852236668a4f7aacd552c0d0ddc3d4. Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 13 20:20:31.565631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:20:31.566645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:20:31.568084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 20:20:31.569061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 20:20:31.569863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5f852236668a4f7aacd552c0d0ddc3d4. Sep 13 20:20:31.569863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 20:20:31.569863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1225/2433] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:20:31 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 31 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 13 20:20:31.664168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:20:31.674107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eb53d2-972d-4630-8457-7bc1e738b32d service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:20:31.679203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:20:31.679793 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:20:31.680072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:20:31.680343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:20:31.688864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:20:31.690337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:20:31.699528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1209/2434] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:20:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:20:41.556375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=37 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:20:41.695617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=67 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:00.601340 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:00.602750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:00.714632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf47409-9d71-481b-9fb2-41f527cddd03 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:00.720155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:00.720757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:00.721033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:00.721305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:00.723597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:00.744113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:00.745689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:00.763903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1226/2435] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:10.761429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=59 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:17.275193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:17.276578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:17.308548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0d4a36-0167-4921-a584-ad86a3d9d53c service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:17.312080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:17.312663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:17.312933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:17.313200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:17.314888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:17.393555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:17.394995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:17.413134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1210/2436] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:27.409948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=50 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:32.797803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:32.799803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:32.874241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ba052-5bad-4345-b896-c1403bc0784e service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:32.878671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:32.879287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:32.879561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:32.879834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:32.881595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:32.951475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:32.952914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:32.963492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1227/2437] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:36.009500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.021641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d08fd6-1845-4779-9517-97a9bfa0d5d8 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.026576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.027174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.027454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.027720 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.029447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:36.046108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.046108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.054933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1211/2438] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:22:36.066632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.068551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:36.083162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ace31-562a-4932-8586-1810a05e100a service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.089562 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.090304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.090583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.090861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] PATH_INFO: `/v3/users/6a68278c8556492497def971fb1674a8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.091998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a68278c8556492497def971fb1674a8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.094039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.145255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a68278c8556492497def971fb1674a8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.152998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a68278c8556492497def971fb1674a8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.153385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a68278c8556492497def971fb1674a8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.154097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a68278c8556492497def971fb1674a8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.161859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d45a090b-d76c-4187-bd58-609733957d11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a68278c8556492497def971fb1674a8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.179179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1228/2439] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:22:36 2022] DELETE /identity/v3/users/6a68278c8556492497def971fb1674a8 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:22:36.342200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.355920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e5a71-08af-4c72-a683-ac033f14055b service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.413483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.414073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.414352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.414623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] PATH_INFO: `/v3/projects/1c5fb3ff52124db488fc164b595503c4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.415982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c5fb3ff52124db488fc164b595503c4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.417923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.463981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c5fb3ff52124db488fc164b595503c4', 'request_id': 'req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.469849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c5fb3ff52124db488fc164b595503c4', 'request_id': 'req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.477304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c5fb3ff52124db488fc164b595503c4', 'request_id': 'req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.487317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] The token cache is being invalidate because project 1c5fb3ff52124db488fc164b595503c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:22:36.487597 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.488210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3bbd9a95-0c17-4ea7-848d-5981c3ae5daa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:22:36.489659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1212/2440] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:22:36 2022] DELETE /identity/v3/projects/1c5fb3ff52124db488fc164b595503c4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:22:36.495399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.495716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.495994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.515173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:22:36.516672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:36.525417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:36.577671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1229/2441] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:22:36.585943 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.586224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.586519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.606298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:22:36.607482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:36.615195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1213/2442] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:22:36.619311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.629500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd337795-fde1-490d-9105-aaf7cbc558b6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.634087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.634684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.634964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.635237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.635875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:22:36.636177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.637800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.649767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1230/2443] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:22:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:22:36.653986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.663347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9c797-0bd1-4058-b45c-8c8dd15d5490 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.667940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.668462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.668739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.669020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.669631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.671550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.697465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1214/2444] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:22:36 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:22:36.701679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.712257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a272f-b009-42b3-9220-6262788478f8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.772810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.773416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.773695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.773999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.774625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.776540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.797150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1231/2445] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:22:36 2022] POST /identity/v3/users => generated 283 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:22:36.801316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.811396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0c2d2a-beb1-47c6-bca7-a29e081bbced admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.816426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.817010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.817283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.817550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.818227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.819796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.829874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1215/2446] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:22:36.834487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.844516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc57db-0772-45c4-b172-13bcf6ac26fa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.849139 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f31de9-e228-4743-94c6-55f9c2ef7f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.849750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f31de9-e228-4743-94c6-55f9c2ef7f98 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.850034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f31de9-e228-4743-94c6-55f9c2ef7f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.850304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f31de9-e228-4743-94c6-55f9c2ef7f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.850938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f31de9-e228-4743-94c6-55f9c2ef7f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.852517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f31de9-e228-4743-94c6-55f9c2ef7f98 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.862884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1232/2447] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:22:36.867094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:36.876997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2135ca-9c0b-4bd1-b1dd-57087cdfcc0a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:36.882158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:36.882786 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.883060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.883335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] PATH_INFO: `/v3/projects/b96f721041cb4484b3436a00794121d2/users/349e7f01d6934b6bba9282e327ca4535/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:36.898320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96f721041cb4484b3436a00794121d2, user_id=349e7f01d6934b6bba9282e327ca4535, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:36.900958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:36.913565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1216/2448] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:22:36 2022] PUT /identity/v3/projects/b96f721041cb4484b3436a00794121d2/users/349e7f01d6934b6bba9282e327ca4535/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:22:36.979039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:36.979240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:36.979516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.007771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] MFA Rules not processed for user `349e7f01d6934b6bba9282e327ca4535`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:22:37.009002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:37.070750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1233/2449] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:22:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:22:37.075111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:37.088119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:37.137779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e5efe-d17b-4e1e-aa1d-515a7f4f1676 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:37.142385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:37.142974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.143246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.143537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.144170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:37.146086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:37.171328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1217/2450] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:22:37 2022] POST /identity/v3/projects => generated 363 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:22:37.175936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:37.187659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4244dadb-eb75-4552-8ae4-d110eac4f82f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:37.192934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:37.244266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:37.244862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.245136 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.245402 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.246051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:37.247934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:37.269909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1234/2451] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:22:37 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:22:37.274058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:37.284590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9183ce08-7e23-4326-9856-4fd42afc3dae admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:37.289362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:37.290054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.290326 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.290590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.291215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:37.292803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:37.303523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1218/2452] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:22:37.307459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:37.317843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047bf221-a38a-462d-a9dd-fcbea8bca525 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:37.322511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:37.323143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.323424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.323698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] PATH_INFO: `/v3/projects/194d52a055f34d2e9558c108120483af/users/84e96eaf2ee24ef0bfd7ba6207d74ec4/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.338177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194d52a055f34d2e9558c108120483af, user_id=84e96eaf2ee24ef0bfd7ba6207d74ec4, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:37.340844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:37.354030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1235/2453] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:22:37 2022] PUT /identity/v3/projects/194d52a055f34d2e9558c108120483af/users/84e96eaf2ee24ef0bfd7ba6207d74ec4/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:22:37.357890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:37.368558 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae595d1-bc60-496e-a6d7-5553aa744cd8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:37.425415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fad589f-d264-4990-a357-5206c028289e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:37.426037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad589f-d264-4990-a357-5206c028289e admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.426308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad589f-d264-4990-a357-5206c028289e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.426575 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fad589f-d264-4990-a357-5206c028289e admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.427206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad589f-d264-4990-a357-5206c028289e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:37.428768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fad589f-d264-4990-a357-5206c028289e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:37.438356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1219/2454] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:22:37.442498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:37.452561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ee0aa-802c-439a-baa2-18c699524f56 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:37.458395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:37.459045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.459325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.459594 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] PATH_INFO: `/v3/projects/194d52a055f34d2e9558c108120483af/users/84e96eaf2ee24ef0bfd7ba6207d74ec4/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.461501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194d52a055f34d2e9558c108120483af, user_id=84e96eaf2ee24ef0bfd7ba6207d74ec4, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:37.464183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:37.475004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1236/2455] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:22:37 2022] PUT /identity/v3/projects/194d52a055f34d2e9558c108120483af/users/84e96eaf2ee24ef0bfd7ba6207d74ec4/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:22:37.484799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:37.485091 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:37.485364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:37.513807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] MFA Rules not processed for user `84e96eaf2ee24ef0bfd7ba6207d74ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:22:37.515009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:37.582100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1220/2456] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:22:37 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:22:47.470773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=40,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:47.578615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:22:52.043435 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:22:52.045200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:52.101541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d3386-8285-4892-b33e-331253831621 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:22:52.105035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:22:52.105625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:22:52.106007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:22:52.106276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:22:52.107969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:22:52.117667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:22:52.119098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71b3bb3-3448-4291-a372-95269dab538e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:22:52.126930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1237/2457] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:22:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:23:02.123481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=26 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:37.652295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:37.714834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a321094-0b02-48c3-9d2d-dd847955f976 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:37.719310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:37.719927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:37.720201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:37.720471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] PATH_INFO: `/v3/users/349e7f01d6934b6bba9282e327ca4535` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:37.721576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=349e7f01d6934b6bba9282e327ca4535)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:37.723519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:37.769928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349e7f01d6934b6bba9282e327ca4535'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:37.777077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349e7f01d6934b6bba9282e327ca4535'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:37.777684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349e7f01d6934b6bba9282e327ca4535'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:37.778332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349e7f01d6934b6bba9282e327ca4535'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:37.785593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349e7f01d6934b6bba9282e327ca4535'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:37.802863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1221/2458] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:23:37 2022] DELETE /identity/v3/users/349e7f01d6934b6bba9282e327ca4535 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:37.807410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:37.807891 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:37.807984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:37.826923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:23:37.828128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:37.885789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1238/2459] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:23:37.895352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:37.896763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:37.954057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18552bd-3470-4c62-9b81-553fe05f93dd admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:37.957503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:37.958123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:37.958396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:37.958660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:37.967422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:37.968822 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:37.977940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1222/2460] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:23:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:23:38.243158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:38.245001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:38.258162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba375c6-4901-4158-972a-9a8f8f074cca None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:38.262904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:38.263533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:38.263807 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:38.264074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] PATH_INFO: `/v3/projects/b96f721041cb4484b3436a00794121d2` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:38.265206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96f721041cb4484b3436a00794121d2)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:38.267382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:38.314690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96f721041cb4484b3436a00794121d2', 'request_id': 'req-32eaea2a-fb1c-431e-9405-bad7c53643c3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.320636 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96f721041cb4484b3436a00794121d2', 'request_id': 'req-32eaea2a-fb1c-431e-9405-bad7c53643c3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.327755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96f721041cb4484b3436a00794121d2', 'request_id': 'req-32eaea2a-fb1c-431e-9405-bad7c53643c3'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.338495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] The token cache is being invalidate because project b96f721041cb4484b3436a00794121d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:23:38.338843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.339558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.341098 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1239/2461] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:23:38 2022] DELETE /identity/v3/projects/b96f721041cb4484b3436a00794121d2 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:38.346463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:38.358219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36abb53b-2173-4563-8b66-79d2eca4f30f service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:38.377120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:38.417297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:38.417937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:38.418213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:38.418481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] PATH_INFO: `/v3/users/84e96eaf2ee24ef0bfd7ba6207d74ec4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:38.419582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84e96eaf2ee24ef0bfd7ba6207d74ec4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:38.421477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:38.466846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84e96eaf2ee24ef0bfd7ba6207d74ec4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.474070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84e96eaf2ee24ef0bfd7ba6207d74ec4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.474683 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84e96eaf2ee24ef0bfd7ba6207d74ec4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.475287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84e96eaf2ee24ef0bfd7ba6207d74ec4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.482174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-330c483e-d429-4ace-a7c4-f2b2cdb6a5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84e96eaf2ee24ef0bfd7ba6207d74ec4'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.499454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1223/2462] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:23:38 2022] DELETE /identity/v3/users/84e96eaf2ee24ef0bfd7ba6207d74ec4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:38.774286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:38.776602 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:38.796799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eaea2a-fb1c-431e-9405-bad7c53643c3 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:38.857424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:38.858077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:38.858350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:38.858621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] PATH_INFO: `/v3/projects/194d52a055f34d2e9558c108120483af` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:38.859750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=194d52a055f34d2e9558c108120483af)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:38.861687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:38.892664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:38.910423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194d52a055f34d2e9558c108120483af', 'request_id': 'req-29f206cc-e403-4803-a300-e736459100fa'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.916345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194d52a055f34d2e9558c108120483af', 'request_id': 'req-29f206cc-e403-4803-a300-e736459100fa'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.923688 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194d52a055f34d2e9558c108120483af', 'request_id': 'req-29f206cc-e403-4803-a300-e736459100fa'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.933963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] The token cache is being invalidate because project 194d52a055f34d2e9558c108120483af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:23:38.934289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.935042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-29f206cc-e403-4803-a300-e736459100fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:23:38.936755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1240/2463] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:23:38 2022] DELETE /identity/v3/projects/194d52a055f34d2e9558c108120483af => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:38.942142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:38.942604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:38.942707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:38.962384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:23:38.963561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:39.027866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1224/2464] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:23:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:23:39.035660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.035944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.036216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.055692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:23:39.056878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:39.065144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1241/2465] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:23:39.069153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.079113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f1f59-0e0e-4e82-9700-dad4ed94ef1d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.083535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.084117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.084390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.084657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.085285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:23:39.085580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.087180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.098320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1225/2466] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:23:39.102590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.111711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e7e2bd-1d19-4b6c-a5d0-3321347d4a68 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.116479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.117066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.117344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.117614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.118275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.120183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.147924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1242/2467] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:23:39.151969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.161853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fc7876-b01a-4e66-be06-fbbbb90607da admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.186431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:39.217266 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.217878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.218151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.218419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.219060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.220938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.241584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1226/2468] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:23:39.245624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.256235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d3c33-a017-406d-905f-52d0ff16dc89 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.260731 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.261313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.261587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.261895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.262529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.264129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.274018 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1243/2469] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:23:39.277931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.288946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd9d2e3-0778-4df2-a210-d2d18c675c2a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.293678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46e6f6-7799-4c19-af0e-a65b9e39dd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.294298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46e6f6-7799-4c19-af0e-a65b9e39dd1f admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.294572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46e6f6-7799-4c19-af0e-a65b9e39dd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.294836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46e6f6-7799-4c19-af0e-a65b9e39dd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.295499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46e6f6-7799-4c19-af0e-a65b9e39dd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.297600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46e6f6-7799-4c19-af0e-a65b9e39dd1f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.308094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1227/2470] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:23:39.312046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.322343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617daa89-e08b-43da-8f06-5b7750df36b7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.326849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.327478 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.327753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.328021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] PATH_INFO: `/v3/projects/5b206df4e4de4586bfc58ac59181d5d3/users/519f0c0e8fcc42498098d855ea46ca39/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.342004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b206df4e4de4586bfc58ac59181d5d3, user_id=519f0c0e8fcc42498098d855ea46ca39, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.344633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.357309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1244/2471] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:23:39 2022] PUT /identity/v3/projects/5b206df4e4de4586bfc58ac59181d5d3/users/519f0c0e8fcc42498098d855ea46ca39/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:39.369133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.369418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.369688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.396928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] MFA Rules not processed for user `519f0c0e8fcc42498098d855ea46ca39`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:23:39.398175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:39.460724 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1228/2472] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:23:39.465309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.524304 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172750e6-c58f-4c5f-9273-67492882e19e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.528882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.529416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.529692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.529992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.530613 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.532481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.558563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1245/2473] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:23:39.563240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.573595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d3188-9fa0-4c26-963f-637432f90cac None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.628497 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.629100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.629376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.629645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.630300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.632229 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.654515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1229/2474] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:23:39.659217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.669598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd52306-6391-4ffe-b882-1d0c707786d6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.674873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.675467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.675744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.676019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.676652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.678286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.689015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1246/2475] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:23:39.692831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.702579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd8051-7dd7-4bdc-b653-93f95622aae5 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.707817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd61012d-9120-40b3-abff-a6d612226744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.708398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61012d-9120-40b3-abff-a6d612226744 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.708674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61012d-9120-40b3-abff-a6d612226744 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.708948 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd61012d-9120-40b3-abff-a6d612226744 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.709577 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61012d-9120-40b3-abff-a6d612226744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.711186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd61012d-9120-40b3-abff-a6d612226744 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.721538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1230/2476] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:23:39.725305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.735660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66f94b4-5e94-479f-9fc7-8f082d3e3240 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.740321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.741085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.741365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.741637 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] PATH_INFO: `/v3/projects/7ce68e85561b4f03b47f0e548df003e4/users/21581c41bed14efd81c40d8d2dd4a18e/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.756003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce68e85561b4f03b47f0e548df003e4, user_id=21581c41bed14efd81c40d8d2dd4a18e, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.758701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.767019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:39.771678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1247/2477] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:23:39 2022] PUT /identity/v3/projects/7ce68e85561b4f03b47f0e548df003e4/users/21581c41bed14efd81c40d8d2dd4a18e/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:39.782284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.782508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.782780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.811338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] MFA Rules not processed for user `21581c41bed14efd81c40d8d2dd4a18e`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:23:39.812541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:39.816472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:39.877440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1231/2478] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:23:39.882054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.943462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e3ed4-3122-43d2-9077-3fc4d8731bb8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:39.948967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:39.949938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:39.950245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:39.950769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:39.951400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:39.953293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:39.979872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1248/2479] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:23:39.983923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:39.995093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8249f115-acaa-444e-add5-ed5c8a69810a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:40.051949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:40.052484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:40.052757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:40.053021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:40.053655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:40.055553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:40.075775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1232/2480] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:23:39 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:23:40.080276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:40.090587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbb1ac9-44bf-4dc4-ba14-8a42f1b2db12 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:40.095673 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:40.096258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:40.096532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:40.096797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:40.097426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:40.099049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:40.109248 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1249/2481] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:23:40.113318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:40.123032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f0fc6a-53f4-425b-becb-c12f91015241 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:40.127522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:40.128150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:40.128429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:40.128691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] PATH_INFO: `/v3/projects/226e2df4024b455a90edbc83bcdb3e3f/users/aaa6806c95594dd6905817bb91e83bd5/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:40.143285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=226e2df4024b455a90edbc83bcdb3e3f, user_id=aaa6806c95594dd6905817bb91e83bd5, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:40.145916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:40.159649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1233/2482] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:23:40 2022] PUT /identity/v3/projects/226e2df4024b455a90edbc83bcdb3e3f/users/aaa6806c95594dd6905817bb91e83bd5/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:40.163899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:40.174412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131b1c8-f497-4617-8ba8-c87587f66539 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:40.229790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110adc1-41a7-45e2-a711-c276551e6ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:40.230396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110adc1-41a7-45e2-a711-c276551e6ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:40.230670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110adc1-41a7-45e2-a711-c276551e6ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:40.230935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110adc1-41a7-45e2-a711-c276551e6ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:40.231587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110adc1-41a7-45e2-a711-c276551e6ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:40.233147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110adc1-41a7-45e2-a711-c276551e6ad1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:40.243021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1250/2483] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:23:40 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:23:40.247421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:23:40.259138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5e8ee1-93e0-4abc-b9a1-75ddac87f308 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:23:40.264000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:23:40.264658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:40.264942 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:40.265217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] PATH_INFO: `/v3/projects/226e2df4024b455a90edbc83bcdb3e3f/users/aaa6806c95594dd6905817bb91e83bd5/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:40.267179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] RBAC: Authorizing `identity:create_grant(project_id=226e2df4024b455a90edbc83bcdb3e3f, user_id=aaa6806c95594dd6905817bb91e83bd5, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:23:40.270165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:23:40.280431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1234/2484] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:23:40 2022] PUT /identity/v3/projects/226e2df4024b455a90edbc83bcdb3e3f/users/aaa6806c95594dd6905817bb91e83bd5/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:23:40.290848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:23:40.291153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:23:40.291446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:23:40.320257 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] MFA Rules not processed for user `aaa6806c95594dd6905817bb91e83bd5`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:23:40.321385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:23:40.379096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:40.389234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1251/2485] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:23:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:23:50.276787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:23:50.386702 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:33.006347 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:33.007993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:33.064538 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ebfab-5e77-431e-8a81-865d179df25a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:33.068043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:33.068626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:33.068898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:33.069162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:33.070851 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] Could not find project: 194d52a055f34d2e9558c108120483af.: keystone.exception.ProjectNotFound: Could not find project: 194d52a055f34d2e9558c108120483af. Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 13 20:24:33.083358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:24:33.084246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:24:33.085641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application with Lock( Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 20:24:33.086662 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 20:24:33.087511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 194d52a055f34d2e9558c108120483af. Sep 13 20:24:33.087511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: ERROR keystone.server.flask.application Sep 13 20:24:33.087511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1235/2486] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:33 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 79 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 13 20:24:33.236216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:33.245671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6b88f-5f42-4a8d-bcf8-7b380b29b0ee None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:33.249831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:33.250424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:33.250702 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:33.250972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:33.259722 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:33.261074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:33.270267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1252/2487] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:24:43.074298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=24 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:43.266215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:24:59.309225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:24:59.310638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:59.379129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8c0a94-178d-44b2-b002-0c375a4280fb service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:24:59.384103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:24:59.384692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:24:59.384967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:24:59.385236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:24:59.387060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:24:59.398132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:24:59.399692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:24:59.410083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1236/2488] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:24:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:09.405212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=32 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:25:57.155574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:57.157169 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:57.168568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cca8ed-64cf-4509-b7b0-553d38a9098b service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:57.172839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:57.173438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:57.173713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:57.174014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:57.175758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:57.242170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:57.243635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:57.253019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1253/2489] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:57.308981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:57.319186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3df37-8a87-441c-8bdc-a585e58191ce service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:57.322663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:57.323255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:57.323527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:57.323795 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:57.325463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:57.382400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:57.383803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:57.392592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1237/2490] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:58.332617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:58.334117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:58.400797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:58.411025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316af499-4bf1-4855-b8ad-5f26b4f47196 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:58.415009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:58.415608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:58.415886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:58.416157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:58.418152 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:58.418388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e747cc-458e-4f7f-ba03-89d8dcb8239b service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:58.423409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:58.424072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:58.424352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:58.424625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:58.426448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] Could not find project: da91a2ee43f6462a8ed96e2a8291d24d.: keystone.exception.ProjectNotFound: Could not find project: da91a2ee43f6462a8ed96e2a8291d24d. Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Sep 13 20:25:58.438233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:25:58.439207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self._validate_token_resources() Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 20:25:58.440075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application with Lock( Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._enter() Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.creator() Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application created_value = creator( Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 20:25:58.441004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 20:25:58.442252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: da91a2ee43f6462a8ed96e2a8291d24d. Sep 13 20:25:58.442252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: ERROR keystone.server.flask.application Sep 13 20:25:58.442252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1254/2491] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:58 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 108 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Sep 13 20:25:58.492653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:58.494104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:58.505227 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1238/2492] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:58.569220 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:58.578872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ae27b-2e42-480e-acbd-ed1994d19314 service glance] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:58.582433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:58.583013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:58.583288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:58.583638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:58.585324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:58.599426 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:58.600844 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:58.611648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1255/2493] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:59.248110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:59.257655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44eded-4b5e-42c3-b1c3-0979ac0c0f58 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:59.261928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:59.262527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.262971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.263405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.337237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:59.339261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:59.349702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1239/2494] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:25:59.475857 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:25:59.477432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:59.492538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035a7b3b-95d4-4128-80c0-8e511c73bcac service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:25:59.498372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:25:59.498978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:25:59.499263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:25:59.499574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:25:59.502360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:25:59.516153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:25:59.517725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:25:59.528787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1256/2495] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:25:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:05.175858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:05.177303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:05.205591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:05.241138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11103369-70c5-4843-8956-0dc656f7baf1 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:05.245202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:05.246112 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:05.246395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:05.246668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:05.248652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:05.316069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:05.317490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:05.327051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1240/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:05.780899 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:05.782331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:05.796524 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d727a936-8fa4-4cdf-be9f-861a14eef154 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:05.801950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:05.802644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:05.802926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:05.803196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:05.804961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:05.817121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:05.818777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:05.832909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:05.836023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:05.840489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1257/2497] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:05.849274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf57b15-7f13-4d34-8c93-abfaa74fe8dc service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:05.853911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:05.854528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:05.854790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:05.855125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] PATH_INFO: `/v3/users/354753f56898466e8f1854745e658ffc` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:05.856239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] RBAC: Authorizing `identity:delete_user(user_id=354753f56898466e8f1854745e658ffc)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:05.858270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:05.904847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '354753f56898466e8f1854745e658ffc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:05.912407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '354753f56898466e8f1854745e658ffc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:05.913166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '354753f56898466e8f1854745e658ffc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:05.913946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '354753f56898466e8f1854745e658ffc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:05.920919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '354753f56898466e8f1854745e658ffc'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:05.938025 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1241/2498] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:05 2022] DELETE /identity/v3/users/354753f56898466e8f1854745e658ffc => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:06.111314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:06.122422 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09fc56-ddea-4d34-a9bc-a71a7629f577 service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:06.174491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:06.187061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:06.187720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:06.187998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:06.188270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] PATH_INFO: `/v3/projects/b9320188c1f743f1bdf221bfa06b6f9a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:06.189372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9320188c1f743f1bdf221bfa06b6f9a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:06.191294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:06.237183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9320188c1f743f1bdf221bfa06b6f9a', 'request_id': 'req-ad3bcd0d-657f-4e7f-8889-f8b828cae479'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:06.243170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9320188c1f743f1bdf221bfa06b6f9a', 'request_id': 'req-ad3bcd0d-657f-4e7f-8889-f8b828cae479'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:06.250266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9320188c1f743f1bdf221bfa06b6f9a', 'request_id': 'req-ad3bcd0d-657f-4e7f-8889-f8b828cae479'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:06.260574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] The token cache is being invalidate because project b9320188c1f743f1bdf221bfa06b6f9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:06.260865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:06.261494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:06.262950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1258/2499] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:06 2022] DELETE /identity/v3/projects/b9320188c1f743f1bdf221bfa06b6f9a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:12.085507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:12.086935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:12.132761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:12.144411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b573f7-a2ca-4d57-a48a-027144a2adec admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:12.192235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:12.192830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:12.193110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:12.193380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:12.195121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:12.251572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:12.252972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80403f6-caee-4643-a930-b11b8d97bb3e service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:12.261922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1242/2500] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:12 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:26:14.380800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:14.382235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:14.446123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3bcd0d-657f-4e7f-8889-f8b828cae479 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:14.450700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:14.451324 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:14.451601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:14.451874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] PATH_INFO: `/v3/users/519f0c0e8fcc42498098d855ea46ca39` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:14.453050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=519f0c0e8fcc42498098d855ea46ca39)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:14.454986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:14.502239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '519f0c0e8fcc42498098d855ea46ca39'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:14.509199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '519f0c0e8fcc42498098d855ea46ca39'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:14.509853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '519f0c0e8fcc42498098d855ea46ca39'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:14.510448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '519f0c0e8fcc42498098d855ea46ca39'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:14.517680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '519f0c0e8fcc42498098d855ea46ca39'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:14.528225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:14.536967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1259/2501] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:14 2022] DELETE /identity/v3/users/519f0c0e8fcc42498098d855ea46ca39 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:14.541385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:14.541876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:14.541988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:14.569298 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:14.570557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:14.628665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1243/2502] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:14.638233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:14.639614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:14.699135 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0576cef-6ba3-46cf-84ed-5d8c3f09e58f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:14.703787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:14.704385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:14.704663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:14.704932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:14.713982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:14.715392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:14.724649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1260/2503] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:14.985044 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:14.986490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:14.998401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb258b-8ccf-48dc-81aa-8ea72fa33956 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:15.003490 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:15.004173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:15.004456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:15.004728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] PATH_INFO: `/v3/projects/5b206df4e4de4586bfc58ac59181d5d3` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:15.005885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b206df4e4de4586bfc58ac59181d5d3)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:15.007789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:15.052428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:15.056561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b206df4e4de4586bfc58ac59181d5d3', 'request_id': 'req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.063525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b206df4e4de4586bfc58ac59181d5d3', 'request_id': 'req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.072339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b206df4e4de4586bfc58ac59181d5d3', 'request_id': 'req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.085622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] The token cache is being invalidate because project 5b206df4e4de4586bfc58ac59181d5d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:15.085937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.086584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.088068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1244/2504] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:14 2022] DELETE /identity/v3/projects/5b206df4e4de4586bfc58ac59181d5d3 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:15.092088 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:15.102747 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef09ded2-9cd9-4203-9af7-839609685ac7 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:15.162593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:15.163122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:15.163394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:15.163663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] PATH_INFO: `/v3/users/21581c41bed14efd81c40d8d2dd4a18e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:15.164781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21581c41bed14efd81c40d8d2dd4a18e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:15.166737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:15.212433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21581c41bed14efd81c40d8d2dd4a18e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.222077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21581c41bed14efd81c40d8d2dd4a18e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.222719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21581c41bed14efd81c40d8d2dd4a18e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.223331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21581c41bed14efd81c40d8d2dd4a18e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.230377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21581c41bed14efd81c40d8d2dd4a18e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.248774 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1261/2505] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:15 2022] DELETE /identity/v3/users/21581c41bed14efd81c40d8d2dd4a18e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:15.518918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:15.519036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:15.534738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d06da-85ac-4028-9cef-2a64e7ad5ad3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:15.592448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:15.593631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:15.593891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:15.594151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] PATH_INFO: `/v3/projects/7ce68e85561b4f03b47f0e548df003e4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:15.595280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce68e85561b4f03b47f0e548df003e4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:15.597187 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:15.643086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce68e85561b4f03b47f0e548df003e4', 'request_id': 'req-c5102cee-822d-4cd1-876a-8c548f6a202c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.649088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce68e85561b4f03b47f0e548df003e4', 'request_id': 'req-c5102cee-822d-4cd1-876a-8c548f6a202c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.657455 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce68e85561b4f03b47f0e548df003e4', 'request_id': 'req-c5102cee-822d-4cd1-876a-8c548f6a202c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.674542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] The token cache is being invalidate because project 7ce68e85561b4f03b47f0e548df003e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:15.674837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.675672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.677156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1245/2506] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:15 2022] DELETE /identity/v3/projects/7ce68e85561b4f03b47f0e548df003e4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:15.683507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:15.684948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:15.697684 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c852b4-5096-4a27-8a9c-858b9d3bf362 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:15.709622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:15.754084 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:15.754710 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:15.754988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:15.755260 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] PATH_INFO: `/v3/users/aaa6806c95594dd6905817bb91e83bd5` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:15.756361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaa6806c95594dd6905817bb91e83bd5)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:15.761716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:15.806017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaa6806c95594dd6905817bb91e83bd5'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.813470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaa6806c95594dd6905817bb91e83bd5'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.814101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaa6806c95594dd6905817bb91e83bd5'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.814711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaa6806c95594dd6905817bb91e83bd5'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.821512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3c26ccb5-8ee3-4443-8197-857f7343bbd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaa6806c95594dd6905817bb91e83bd5'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:15.838898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1262/2507] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:15 2022] DELETE /identity/v3/users/aaa6806c95594dd6905817bb91e83bd5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:16.118975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.120432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:16.134544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5102cee-822d-4cd1-876a-8c548f6a202c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.199373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.200001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.200307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.200582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] PATH_INFO: `/v3/projects/226e2df4024b455a90edbc83bcdb3e3f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.201700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] RBAC: Authorizing `identity:delete_project(project_id=226e2df4024b455a90edbc83bcdb3e3f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.204262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.206496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:16.251026 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226e2df4024b455a90edbc83bcdb3e3f', 'request_id': 'req-4e40dcd4-cbe6-400f-b338-c49c66dd509a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:16.256998 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226e2df4024b455a90edbc83bcdb3e3f', 'request_id': 'req-4e40dcd4-cbe6-400f-b338-c49c66dd509a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:16.264190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226e2df4024b455a90edbc83bcdb3e3f', 'request_id': 'req-4e40dcd4-cbe6-400f-b338-c49c66dd509a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:16.275363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] The token cache is being invalidate because project 226e2df4024b455a90edbc83bcdb3e3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:16.275705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:16.276342 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e40dcd4-cbe6-400f-b338-c49c66dd509a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:16.278030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1246/2508] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:16 2022] DELETE /identity/v3/projects/226e2df4024b455a90edbc83bcdb3e3f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:16.284225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.284587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.284892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.340674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:16.341949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:16.403469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1263/2509] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:16.412007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.412354 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.412687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.433018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:16.434239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:16.442380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1247/2510] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:16.446454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.456563 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c6997-aca6-48bb-9eb7-87b3c234e798 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.461033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.461623 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.461952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.462226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.462862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:16.463164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.464754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.473187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:16.476557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1264/2511] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:16.480773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.489652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdddc77a-9510-4e22-b8c4-73aafb1c8280 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.494241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.494841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.495110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.495375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.495989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.497861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.534493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1248/2512] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:16.538470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.548314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf552a7-805c-43b1-a523-8538ae75299c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.605692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.606341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.606621 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.606892 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.607510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.609925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.630798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1265/2513] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/users => generated 283 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:16.634925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.645221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6edf1a-0cdf-4060-9e60-2e83a6ff0543 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.649799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.650389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.650667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.650937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.651574 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.653129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.663040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1249/2514] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:16.666880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.677263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20167411-9aba-492b-a891-7937ae2c0c4b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.681980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e1e8d-411f-4a50-88d5-f67cb2aef68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.682567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e1e8d-411f-4a50-88d5-f67cb2aef68c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.682846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e1e8d-411f-4a50-88d5-f67cb2aef68c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.683119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e1e8d-411f-4a50-88d5-f67cb2aef68c admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.683754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e1e8d-411f-4a50-88d5-f67cb2aef68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.685320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e1e8d-411f-4a50-88d5-f67cb2aef68c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.696439 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1266/2515] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:16.701165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.711068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0bb2d-08f6-4c7e-9781-cffd56bf1f10 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.715633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.716262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.716542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.716806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] PATH_INFO: `/v3/projects/ab0138be4bde45139cacf47e5e117fe8/users/aee23f5a447d4013883c9bb57d3525f2/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.730907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab0138be4bde45139cacf47e5e117fe8, user_id=aee23f5a447d4013883c9bb57d3525f2, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.733517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.746159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1250/2516] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:16 2022] PUT /identity/v3/projects/ab0138be4bde45139cacf47e5e117fe8/users/aee23f5a447d4013883c9bb57d3525f2/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:16.757861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.758144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.758415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.786980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] MFA Rules not processed for user `aee23f5a447d4013883c9bb57d3525f2`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:16.788156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:16.853806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1267/2517] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:16.858225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.918949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d5b6c-075d-48d0-864a-d100c9e3469c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:16.924179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:16.924767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:16.925043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:16.925311 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:16.925966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:16.927832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:16.955756 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1251/2518] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:16.959775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:16.971259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e24617-f39a-4554-8765-8f7631677d2a None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.025907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.026501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.026778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.027046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.027662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.029546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.046879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:17.053622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1268/2519] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:16 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:17.057314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:17.063768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:17.067761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c717011-0a97-4395-bfdf-a80b27ef3f7a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.073560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.074327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.074764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.075084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.075955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.077543 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.088203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1252/2520] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:17.092194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:17.102663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669107e-a3f4-401e-91af-6d1d25a6075c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.107309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.107945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.108226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.108498 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] PATH_INFO: `/v3/projects/d8f73907623f486b893fb82c8af76ea3/users/d8ffce2022a74d5c9729e2d70132a13b/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.126073 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f73907623f486b893fb82c8af76ea3, user_id=d8ffce2022a74d5c9729e2d70132a13b, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.128729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.141640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1269/2521] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:17 2022] PUT /identity/v3/projects/d8f73907623f486b893fb82c8af76ea3/users/d8ffce2022a74d5c9729e2d70132a13b/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:17.145296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:17.155063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38cdef-f1ae-4f62-bb80-edf6d73d9d8a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.219190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1daad22-70db-4093-995e-115651e7f506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.219787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daad22-70db-4093-995e-115651e7f506 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.220065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daad22-70db-4093-995e-115651e7f506 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.220331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daad22-70db-4093-995e-115651e7f506 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.220961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daad22-70db-4093-995e-115651e7f506 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.222605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1daad22-70db-4093-995e-115651e7f506 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.231947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1253/2522] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:17 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:17.236596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:17.246698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57774809-dbf5-4073-a22e-5870b884669a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.251384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.252190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.252470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.252744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] PATH_INFO: `/v3/projects/d8f73907623f486b893fb82c8af76ea3/users/d8ffce2022a74d5c9729e2d70132a13b/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.254696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8f73907623f486b893fb82c8af76ea3, user_id=d8ffce2022a74d5c9729e2d70132a13b, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.257315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.267291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1270/2523] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:17 2022] PUT /identity/v3/projects/d8f73907623f486b893fb82c8af76ea3/users/d8ffce2022a74d5c9729e2d70132a13b/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:17.278270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.278557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.278829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.307194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] MFA Rules not processed for user `d8ffce2022a74d5c9729e2d70132a13b`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:17.308340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:17.385456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1254/2524] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:17 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:17.398822 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:17.400525 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:17.457827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99740316-453a-47cc-9d7d-9e881842618f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.461443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.462072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.462353 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.462626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.472075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.473494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.482801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1271/2525] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:17 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:17.564086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:17.573633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6dcb5-12e6-481f-93b0-23f907c2c2ec None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:17.578542 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:17.579204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:17.579545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:17.579881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:17.637951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:17.639370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:17.648680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1255/2526] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:17 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:23.496882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:23.496882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:23.581094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b829658-1208-4d94-be50-4e8e17465e71 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:23.585724 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:23.586462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:23.586804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:23.587147 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] PATH_INFO: `/v3/users/b6deb6f1f00140e9ac92c9255b0ed71e` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:23.588351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6deb6f1f00140e9ac92c9255b0ed71e)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:23.590859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:23.607420 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:23.619188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eacdd-3763-405b-b1db-d63035522de6 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:23.625500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:23.626399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:23.626726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:23.627089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] PATH_INFO: `/v3/users/aee23f5a447d4013883c9bb57d3525f2` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:23.628696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aee23f5a447d4013883c9bb57d3525f2)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:23.631008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:23.636740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:23.646178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6deb6f1f00140e9ac92c9255b0ed71e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.654565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6deb6f1f00140e9ac92c9255b0ed71e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.655218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6deb6f1f00140e9ac92c9255b0ed71e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.656416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6deb6f1f00140e9ac92c9255b0ed71e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.667006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-167c542f-65eb-4538-987f-65631563b19f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6deb6f1f00140e9ac92c9255b0ed71e'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.669180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:23.688908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aee23f5a447d4013883c9bb57d3525f2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.693777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1272/2527] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:23 2022] DELETE /identity/v3/users/b6deb6f1f00140e9ac92c9255b0ed71e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:23.695819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:23.696259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:23.696668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:23.696778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aee23f5a447d4013883c9bb57d3525f2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.697704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aee23f5a447d4013883c9bb57d3525f2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.698341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aee23f5a447d4013883c9bb57d3525f2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.707302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-df3ecbbc-099a-4c42-9307-984a3254cf91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aee23f5a447d4013883c9bb57d3525f2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:23.721912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:23.723329 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:23.725395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1256/2528] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:23 2022] DELETE /identity/v3/users/aee23f5a447d4013883c9bb57d3525f2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:23.730491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:23.730491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:23.730491 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:23.749665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:23.750887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:23.785664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1273/2529] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:23.794796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:23.797441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:23.820780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1257/2530] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:23.841695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:23.843771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:23.860685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f739fe8f-5155-45e4-9592-b59d33884cb8 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:23.864939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:23.866141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:23.866141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:23.866374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:23.876356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:23.877804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:23.888203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1274/2531] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:26:23.907148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c189fd-b2a2-4d72-93dd-803f56ec06ea None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:23.911475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:23.912686 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:23.913123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:23.913823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:23.926821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:23.928874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:23.940161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1258/2532] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:26:24.398916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.400611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.438172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4119c49b-786e-4151-84a8-30fcb884c2f1 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.440099 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.444691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.448477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.449559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.450295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.450379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] PATH_INFO: `/v3/projects/93254702586648759994449e9d63bfd5` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.451834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93254702586648759994449e9d63bfd5)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.454410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.488978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5be25b-1672-46f3-9fd1-c7cea41339f6 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.501293 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.507465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.507859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.508413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] PATH_INFO: `/v3/projects/ab0138be4bde45139cacf47e5e117fe8` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.515826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab0138be4bde45139cacf47e5e117fe8)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.517914 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.534444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93254702586648759994449e9d63bfd5', 'request_id': 'req-5bd8d980-d760-4b0e-b22a-96a111109eb9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.549948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93254702586648759994449e9d63bfd5', 'request_id': 'req-5bd8d980-d760-4b0e-b22a-96a111109eb9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.562449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93254702586648759994449e9d63bfd5', 'request_id': 'req-5bd8d980-d760-4b0e-b22a-96a111109eb9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.567081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:24.579750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] The token cache is being invalidate because project 93254702586648759994449e9d63bfd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:24.580174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.580910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5bd8d980-d760-4b0e-b22a-96a111109eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.582732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1275/2533] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:24 2022] DELETE /identity/v3/projects/93254702586648759994449e9d63bfd5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:24.588363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729487bd-5138-47ea-8fe2-30535e289119 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.593866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729487bd-5138-47ea-8fe2-30535e289119 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.594291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729487bd-5138-47ea-8fe2-30535e289119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.602419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab0138be4bde45139cacf47e5e117fe8', 'request_id': 'req-4b9b3759-273e-412a-b84d-0f7f67c672f5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.611833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab0138be4bde45139cacf47e5e117fe8', 'request_id': 'req-4b9b3759-273e-412a-b84d-0f7f67c672f5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.631799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab0138be4bde45139cacf47e5e117fe8', 'request_id': 'req-4b9b3759-273e-412a-b84d-0f7f67c672f5'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.651465 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-729487bd-5138-47ea-8fe2-30535e289119 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:24.652935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-729487bd-5138-47ea-8fe2-30535e289119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.676905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] The token cache is being invalidate because project ab0138be4bde45139cacf47e5e117fe8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:24.677431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.678330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.678520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:24.685561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1259/2534] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:24 2022] DELETE /identity/v3/projects/ab0138be4bde45139cacf47e5e117fe8 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:24.687120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.688958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.789472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1276/2535] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:24.796413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9b3759-273e-412a-b84d-0f7f67c672f5 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.798001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.798431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.798814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.801763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.802546 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.802908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.803244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] PATH_INFO: `/v3/users/d8ffce2022a74d5c9729e2d70132a13b` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.804439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8ffce2022a74d5c9729e2d70132a13b)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.806465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.824264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:24.825991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:24.845369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1277/2536] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:24.852633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.867114 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30decadd-d458-4c22-b7fe-48f9915d16bc None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.873708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.875830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.875830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.876123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.876970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:24.877277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.878926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.887190 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8ffce2022a74d5c9729e2d70132a13b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.891469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1278/2537] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:24.896419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.904501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8ffce2022a74d5c9729e2d70132a13b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.905254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8ffce2022a74d5c9729e2d70132a13b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.906164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8ffce2022a74d5c9729e2d70132a13b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.915737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8ffce2022a74d5c9729e2d70132a13b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:24.916854 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289cb04-c94e-43c5-a7af-f2167bffe93f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:24.923004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:24.923622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:24.923907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:24.924179 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:24.924815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:24.926773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:24.946241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1260/2538] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:24 2022] DELETE /identity/v3/users/d8ffce2022a74d5c9729e2d70132a13b => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:24.975776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1279/2539] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:24 2022] POST /identity/v3/projects => generated 375 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:24.977179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:24.978641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.026454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:25.071833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e209971-878c-4f89-96df-f4d9adc6edb8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.147397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.148185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.148899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.149178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.149883 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.152307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.180072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1261/2540] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:24 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:25.184407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.198561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a68bea-f57b-4bcb-baba-9a18a0a0d6ac admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.203876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.204250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.204629 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.205002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.205795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.207495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.219868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1280/2541] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:25.223646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.236307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9c780c-d61a-4c51-a401-af33f4f21dbd admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.241640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.242399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.242900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.243283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.244037 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.245806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.262371 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1262/2542] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:25.266881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.278952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c106b3-d7d0-453a-b6f7-69a0d2aaa0e6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.286530 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355d2c9-bae3-40be-9051-6637f156f877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.287690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355d2c9-bae3-40be-9051-6637f156f877 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.288204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355d2c9-bae3-40be-9051-6637f156f877 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.288551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355d2c9-bae3-40be-9051-6637f156f877 admin admin] PATH_INFO: `/v3/projects/0d4afcdd7ba94aba95af66c6ebc8be94/users/edd812186cdf4c33803837796c6cc3da/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.301105 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.304583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.311393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355d2c9-bae3-40be-9051-6637f156f877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d4afcdd7ba94aba95af66c6ebc8be94, user_id=edd812186cdf4c33803837796c6cc3da, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.314327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355d2c9-bae3-40be-9051-6637f156f877 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.318296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c6d5a-d447-4b24-aafd-c968017502f3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.324308 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.325454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.326383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.326846 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] PATH_INFO: `/v3/projects/d8f73907623f486b893fb82c8af76ea3` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.328364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8f73907623f486b893fb82c8af76ea3)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.330545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1281/2543] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:25 2022] PUT /identity/v3/projects/0d4afcdd7ba94aba95af66c6ebc8be94/users/edd812186cdf4c33803837796c6cc3da/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:25.331330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.341954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.342244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.342523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.379443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] MFA Rules not processed for user `edd812186cdf4c33803837796c6cc3da`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:25.381035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.411674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8f73907623f486b893fb82c8af76ea3', 'request_id': 'req-904fe29d-dfc1-440d-aeef-0e728f0373e2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.419209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8f73907623f486b893fb82c8af76ea3', 'request_id': 'req-904fe29d-dfc1-440d-aeef-0e728f0373e2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.430800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8f73907623f486b893fb82c8af76ea3', 'request_id': 'req-904fe29d-dfc1-440d-aeef-0e728f0373e2'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.449588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] The token cache is being invalidate because project d8f73907623f486b893fb82c8af76ea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:25.450370 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.451098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-904fe29d-dfc1-440d-aeef-0e728f0373e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:25.456616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1263/2544] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:25 2022] DELETE /identity/v3/projects/d8f73907623f486b893fb82c8af76ea3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:25.458075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a30b-e0ea-4ada-87e0-7e39dbd13cb2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.458762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a30b-e0ea-4ada-87e0-7e39dbd13cb2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.458762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605a30b-e0ea-4ada-87e0-7e39dbd13cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.477222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1282/2545] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:25.482028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.482916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-a605a30b-e0ea-4ada-87e0-7e39dbd13cb2 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:25.484749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a605a30b-e0ea-4ada-87e0-7e39dbd13cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.485398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.526730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:25.531034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:25.560337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1264/2546] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:25.568199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.568477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.568743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.584690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7671a84-1f2e-42f0-890c-406b35bb1f60 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.589544 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:25.590041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.590771 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.590872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:25.591112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.591448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.592138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.594747 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.599957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1265/2547] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:25.604160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.613397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba762dd7-233c-40c5-bf58-110f017c5a8f None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.618738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.619617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.619891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.620158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.620824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:25.621124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.622744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.630132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1283/2548] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:25.631200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.639434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1266/2549] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:26:25.640912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.652007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4693257c-8251-441e-b183-651095f981a3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.710948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e94bb2-47c5-4a43-b3d9-bcbf9fde50da admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.723785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.724607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.724873 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.725151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.729605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.732691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.766942 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1284/2550] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/users => generated 288 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:25.768310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.769033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.769639 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.770436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.770863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.771072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.773197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.782740 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263e4af8-2246-4da0-a4fc-32e0ad8ff9e0 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.789258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.789424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.789644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.789969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.790619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.792218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.807032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1285/2551] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:25.808643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.817220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1267/2552] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:25.821083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b236142-cf5d-4ffa-8bb1-7db96906c48f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.822649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.910856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fde83-10c0-4a3e-b4de-34be17252da6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.920202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.920366 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.921004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.921377 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.921591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.921689 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.922186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.922708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.922961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.923373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:25.925119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.925233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:25.940606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1286/2553] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:25.944375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.970271 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7adbbd3-50a2-431a-9f22-ed1ddf22aa00 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.977590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0385ad-7aca-4a02-b2cb-3e2f7aadc189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:25.978308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0385ad-7aca-4a02-b2cb-3e2f7aadc189 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:25.978587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0385ad-7aca-4a02-b2cb-3e2f7aadc189 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:25.978887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0385ad-7aca-4a02-b2cb-3e2f7aadc189 admin admin] PATH_INFO: `/v3/projects/38687cd8068d4d02a514442609e67003/users/3422881e81344497a286a44e8af74d52/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:25.979503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1268/2554] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:25 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:25.983391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:25.993600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1090f-adaa-4324-9bba-889dcb3a99b4 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:25.999671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.000477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.000828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.001161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.001992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.003668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.004176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0385ad-7aca-4a02-b2cb-3e2f7aadc189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38687cd8068d4d02a514442609e67003, user_id=3422881e81344497a286a44e8af74d52, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.007216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0385ad-7aca-4a02-b2cb-3e2f7aadc189 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.015701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1269/2555] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:26.018980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.019913 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.029785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1287/2556] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:25 2022] PUT /identity/v3/projects/38687cd8068d4d02a514442609e67003/users/3422881e81344497a286a44e8af74d52/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:26.033922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebadfa0-f3f0-4426-aae3-256221015f4c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.041533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.042035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.042447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.058411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.080941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] MFA Rules not processed for user `3422881e81344497a286a44e8af74d52`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:26.082591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.098403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.099013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.099291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.099562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.100205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.102242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.114954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1270/2557] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:26.118714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.130804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0d8958-a6c2-4e9b-9214-1bcb73e395b7 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.136644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d0800f-d58a-4cb1-b7c5-2c2ed4b65ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.137316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0800f-d58a-4cb1-b7c5-2c2ed4b65ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.137592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0800f-d58a-4cb1-b7c5-2c2ed4b65ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.138016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d0800f-d58a-4cb1-b7c5-2c2ed4b65ec8 admin admin] PATH_INFO: `/v3/projects/68e10d92218c4caf927ea93979e07b23/users/b11b92357b9c4efc8b2f70c11552828d/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.155068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0800f-d58a-4cb1-b7c5-2c2ed4b65ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e10d92218c4caf927ea93979e07b23, user_id=b11b92357b9c4efc8b2f70c11552828d, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.158859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d0800f-d58a-4cb1-b7c5-2c2ed4b65ec8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.161519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1288/2558] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:26.166471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.176855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1271/2559] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:26 2022] PUT /identity/v3/projects/68e10d92218c4caf927ea93979e07b23/users/b11b92357b9c4efc8b2f70c11552828d/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:26.177511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0952e4-1116-422e-adfa-a7dceae628ba None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.189080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.189478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.189859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.221388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] MFA Rules not processed for user `b11b92357b9c4efc8b2f70c11552828d`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:26.222632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.242034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.242644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.242924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.243214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.243824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.245725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.284662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1289/2560] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/projects => generated 375 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:26.286523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.293787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1272/2561] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:26.299362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.313323 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b840f0e2-1d55-4414-90dc-62b961780dcd None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.363924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ffeaf0-73f8-40c6-9f03-a9127e23ae23 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.368994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.369605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.369912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.370182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.370816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.372912 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.379552 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.380286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.380684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.381061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.381826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.384014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.403227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1290/2562] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/users => generated 294 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:26.404407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.412167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.418307 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cf6fa6-93fe-4dcb-ba3b-26f31ade3a43 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.426030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1273/2563] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:26.427206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.428036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.428691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.428967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.429233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.429911 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.431513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.442920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.443727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1291/2564] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:26.447883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.461403 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7894280-a0d2-4f33-8544-ec6a379db72f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.514836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5989d9-ae02-4c64-85f6-f7f0783b52af admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.520431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.521207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.521524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.522302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.522903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.525578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.531674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.532413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.532758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.533104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] PATH_INFO: `/v3/projects/16e295732a1146b9a62bb6787db4bb00/users/564ae28615a14b14a4be75f5d9d04713/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.551607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e295732a1146b9a62bb6787db4bb00, user_id=564ae28615a14b14a4be75f5d9d04713, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.553775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1274/2565] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:26.554502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.557393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.568996 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bd65c-1616-40bd-93b5-c5f3e25895bf admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.570503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1292/2566] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:26 2022] PUT /identity/v3/projects/16e295732a1146b9a62bb6787db4bb00/users/564ae28615a14b14a4be75f5d9d04713/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:26.576543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.632710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.633265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.633500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.633796 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.634439 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.636033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.641529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383c432-f475-4a3e-9979-cf9e9f82a306 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.645641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1275/2567] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:26.646187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.646859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.647143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.647416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.648057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.649553 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.650075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.659591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05290164-2bef-4cd5-9337-5553cfbe70d5 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.660727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1293/2568] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:26.664426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4747cf68-106f-4272-98d1-846bc39e4e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.664711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.665094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747cf68-106f-4272-98d1-846bc39e4e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.665395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747cf68-106f-4272-98d1-846bc39e4e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.665665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4747cf68-106f-4272-98d1-846bc39e4e0f admin admin] PATH_INFO: `/v3/projects/4904823a1905423a94c0e782672738c8/users/78e2084107e944459a2ba3b4e0718a1c/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.674859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42f8f2-9606-461e-8c1f-6f5dfe01caec admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.679578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.680217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.680491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.680762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] PATH_INFO: `/v3/projects/16e295732a1146b9a62bb6787db4bb00/users/564ae28615a14b14a4be75f5d9d04713/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.682682 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747cf68-106f-4272-98d1-846bc39e4e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4904823a1905423a94c0e782672738c8, user_id=78e2084107e944459a2ba3b4e0718a1c, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.682810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16e295732a1146b9a62bb6787db4bb00, user_id=564ae28615a14b14a4be75f5d9d04713, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.685290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4747cf68-106f-4272-98d1-846bc39e4e0f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.685396 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.696979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1294/2569] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:26 2022] PUT /identity/v3/projects/16e295732a1146b9a62bb6787db4bb00/users/564ae28615a14b14a4be75f5d9d04713/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:26.699749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1276/2570] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:26 2022] PUT /identity/v3/projects/4904823a1905423a94c0e782672738c8/users/78e2084107e944459a2ba3b4e0718a1c/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:26.703454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.709845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.710129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.710442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.738341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] MFA Rules not processed for user `564ae28615a14b14a4be75f5d9d04713`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:26.739539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.765404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.770292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8175b02-fc5a-4194-beaf-799f3d22a184 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.830692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:26.840158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.840827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.841122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.841381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.842154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.844332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.846283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1277/2571] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:26.847559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.849778 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.856535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1295/2572] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:26.861612 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:26.872107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2316c-55a0-41bd-bbd5-c9b949082d73 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.877321 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458080f-fa62-4a87-8563-5a4a6370b9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.878386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458080f-fa62-4a87-8563-5a4a6370b9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.878663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458080f-fa62-4a87-8563-5a4a6370b9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.878933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458080f-fa62-4a87-8563-5a4a6370b9e1 admin admin] PATH_INFO: `/v3/projects/4904823a1905423a94c0e782672738c8/users/78e2084107e944459a2ba3b4e0718a1c/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.881499 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458080f-fa62-4a87-8563-5a4a6370b9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4904823a1905423a94c0e782672738c8, user_id=78e2084107e944459a2ba3b4e0718a1c, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:26.884795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458080f-fa62-4a87-8563-5a4a6370b9e1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:26.896529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1296/2573] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:26 2022] PUT /identity/v3/projects/4904823a1905423a94c0e782672738c8/users/78e2084107e944459a2ba3b4e0718a1c/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:26.907093 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.907509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.907825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:26.920145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e79524f-78a8-4e7d-81fb-960b571e987c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:26.938199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] MFA Rules not processed for user `78e2084107e944459a2ba3b4e0718a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:26.939442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:26.972872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:26.973482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:26.973898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:26.974151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.011603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1297/2574] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:26 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:27.022951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:27.024527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:27.033681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.035141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.036743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de895eb-32c7-419d-830c-78cf3005be51 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:27.040313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:27.040926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:27.041172 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:27.041436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:27.044211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1278/2575] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:27.098958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:27.100374 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:27.109535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1298/2576] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:26:28.559269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:28.567693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac7da0-aef9-4cac-ac04-31d578f311d7 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:28.571210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:28.571787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:28.572061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:28.572327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:28.628117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:28.629489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:28.638241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1279/2577] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:29.213044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:29.222584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa031eb-3ae6-416e-883d-43afe6387532 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:29.226944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:29.227644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:29.227989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:29.228328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:29.237814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:29.295235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:29.297249 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:29.307477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1299/2578] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:29 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:30.161315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.162766 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:30.172922 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:30.219045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114e31b-7800-4853-9a5e-371d457b4cd1 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.222584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.223173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.223445 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.223740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.225435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:30.286080 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.287427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff143a4-f656-42ff-9e19-3232ae6e7f43 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.298145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1280/2579] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:30.755615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:30.773664 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc03be2-d45e-4a5e-98cc-0db587a4fa81 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:30.787450 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:30.788278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.788628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.788971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] PATH_INFO: `/v3/users/b11b92357b9c4efc8b2f70c11552828d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.790214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b11b92357b9c4efc8b2f70c11552828d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:30.792361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:30.870835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b11b92357b9c4efc8b2f70c11552828d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.880879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b11b92357b9c4efc8b2f70c11552828d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.881980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b11b92357b9c4efc8b2f70c11552828d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.882574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b11b92357b9c4efc8b2f70c11552828d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.891930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b11b92357b9c4efc8b2f70c11552828d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:30.918319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1300/2580] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:30 2022] DELETE /identity/v3/users/b11b92357b9c4efc8b2f70c11552828d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:30.924494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:30.924931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:30.925283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:30.948697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:30.950169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.042678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1281/2581] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:31.053104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.054739 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.112744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dc657b-ffb4-4e23-b78c-a34d21a1f650 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.116399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.117006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.117284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.117555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.126846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.128275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.138025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1301/2582] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:31.426837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.428519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:31.472992 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a78ad49-47cf-4155-8710-8d0c4928f80c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.480285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.481059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.481428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.481816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] PATH_INFO: `/v3/projects/68e10d92218c4caf927ea93979e07b23` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.483548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e10d92218c4caf927ea93979e07b23)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.486746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.538813 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e10d92218c4caf927ea93979e07b23', 'request_id': 'req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.540989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:31.546075 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e10d92218c4caf927ea93979e07b23', 'request_id': 'req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.558058 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e10d92218c4caf927ea93979e07b23', 'request_id': 'req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.572715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] The token cache is being invalidate because project 68e10d92218c4caf927ea93979e07b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:31.573085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.573825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.575447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1282/2583] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:31 2022] DELETE /identity/v3/projects/68e10d92218c4caf927ea93979e07b23 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:31.580207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:31.589993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd725a1-80b0-4ad0-a337-a4387553f6ea service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:31.607001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:31.651076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:31.651656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:31.651932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:31.652203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] PATH_INFO: `/v3/users/78e2084107e944459a2ba3b4e0718a1c` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:31.653326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e2084107e944459a2ba3b4e0718a1c)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:31.655344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:31.709778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e2084107e944459a2ba3b4e0718a1c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.717529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e2084107e944459a2ba3b4e0718a1c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.718190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e2084107e944459a2ba3b4e0718a1c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.718849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e2084107e944459a2ba3b4e0718a1c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.725898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e87e6f9d-dfb6-4ab3-8151-6d6e73ed95da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e2084107e944459a2ba3b4e0718a1c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:31.744462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1302/2584] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:31 2022] DELETE /identity/v3/users/78e2084107e944459a2ba3b4e0718a1c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:32.155550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.157427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.179829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa4aa52-e3be-4949-bb63-9fc1024d82d8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.294502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.295407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.295905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.296335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] PATH_INFO: `/v3/projects/4904823a1905423a94c0e782672738c8` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.297708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4904823a1905423a94c0e782672738c8)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.302386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.439985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4904823a1905423a94c0e782672738c8', 'request_id': 'req-2c3d12f5-43fd-4c47-89d6-f79143800533'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.447509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4904823a1905423a94c0e782672738c8', 'request_id': 'req-2c3d12f5-43fd-4c47-89d6-f79143800533'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.458521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4904823a1905423a94c0e782672738c8', 'request_id': 'req-2c3d12f5-43fd-4c47-89d6-f79143800533'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.473584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] The token cache is being invalidate because project 4904823a1905423a94c0e782672738c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:32.473906 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.475376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-2c3d12f5-43fd-4c47-89d6-f79143800533 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:32.477203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1283/2585] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:32 2022] DELETE /identity/v3/projects/4904823a1905423a94c0e782672738c8 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:32.482729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.483752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.484045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.521270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:32.522620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.619201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1303/2586] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:32.627620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.627910 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.628191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.690013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:32.692565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:32.706174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1284/2587] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:32.710720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.724264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf7d845-814d-4914-a0d4-fd916f9652eb None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.731481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.732323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.732874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.734228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.735966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:32.736693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.739460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.758952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1304/2588] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:32.763428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.773904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbf597d-3591-4075-b958-29aa3941b739 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.778924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.779535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.780316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.780881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.781530 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.783730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.821823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1285/2589] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:32 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:32.825999 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.848295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0006cac4-b2e6-433e-86e7-1075c8491e08 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.878946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:32.921851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.922535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.922919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.923291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.924038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.926140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:32.954399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1305/2590] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:32 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:32.956595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:32.972849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fe0941-edd6-4c67-8f86-0c7e256d00b9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:32.994209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:32.994866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:32.995144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:32.995413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:32.996100 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:32.997692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.016333 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1286/2591] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:33.020528 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.031110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d5346-1278-4b7d-94c6-c8a5a8edd25d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.036642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb28126d-d607-425c-9b00-d5f786005a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.037285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28126d-d607-425c-9b00-d5f786005a10 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.037564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28126d-d607-425c-9b00-d5f786005a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.037956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb28126d-d607-425c-9b00-d5f786005a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.038627 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28126d-d607-425c-9b00-d5f786005a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.040285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb28126d-d607-425c-9b00-d5f786005a10 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.055047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1306/2592] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:33.056684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.068096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2390fc-dacb-470f-a61e-eaae04059c75 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.073174 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.074005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.074351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.074693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] PATH_INFO: `/v3/projects/cd39a0a036eb47cabfdd1840a9355322/users/2569b0dabcc146bba73908b14427a0bb/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.214063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd39a0a036eb47cabfdd1840a9355322, user_id=2569b0dabcc146bba73908b14427a0bb, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.216679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.230873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1287/2593] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:33 2022] PUT /identity/v3/projects/cd39a0a036eb47cabfdd1840a9355322/users/2569b0dabcc146bba73908b14427a0bb/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:33.242705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.242995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.243298 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.271348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] MFA Rules not processed for user `2569b0dabcc146bba73908b14427a0bb`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:33.272568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.340887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1307/2594] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:33.351742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.357656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.375626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:33.437954 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fa44b-7054-4eb3-bf73-44b30ebb426d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.445665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456d598-a3a0-4995-9512-2af2ed678f6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.446338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d598-a3a0-4995-9512-2af2ed678f6a service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.447043 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d598-a3a0-4995-9512-2af2ed678f6a service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.447318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456d598-a3a0-4995-9512-2af2ed678f6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.461012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456d598-a3a0-4995-9512-2af2ed678f6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.462843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456d598-a3a0-4995-9512-2af2ed678f6a service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.476409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1288/2595] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:33.534212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:33.538070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:33.637550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70afb8d6-4f97-4a97-b52d-0eb8b874b5e5 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:33.643425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:33.644767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.645045 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.645317 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] PATH_INFO: `/v3/users/edd812186cdf4c33803837796c6cc3da` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.646717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=edd812186cdf4c33803837796c6cc3da)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:33.649363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:33.704981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:33.725527 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edd812186cdf4c33803837796c6cc3da'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.741847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edd812186cdf4c33803837796c6cc3da'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.743046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edd812186cdf4c33803837796c6cc3da'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.744094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edd812186cdf4c33803837796c6cc3da'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.760384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edd812186cdf4c33803837796c6cc3da'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:33.821178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1308/2596] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:33 2022] DELETE /identity/v3/users/edd812186cdf4c33803837796c6cc3da => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:33.825472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:33.825827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:33.826102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:33.875765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:33.876979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.036928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1289/2597] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:34.054131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.056019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.115929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.118557 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.157468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b727e310-95d5-4497-a11e-6c8368e3c61a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.162387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e47cc0-50b8-447f-8292-c579c2aa7ed7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.166137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.166801 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.167140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.167479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.168841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.169779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.170214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.170669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] PATH_INFO: `/v3/users/eff856b09e5944a9bdcf7374cd9d4a2a` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.172555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eff856b09e5944a9bdcf7374cd9d4a2a)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.174750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.181896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.183413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.198756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1309/2598] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:34.245987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eff856b09e5944a9bdcf7374cd9d4a2a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.260249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eff856b09e5944a9bdcf7374cd9d4a2a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.261028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eff856b09e5944a9bdcf7374cd9d4a2a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.262959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eff856b09e5944a9bdcf7374cd9d4a2a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.272380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eff856b09e5944a9bdcf7374cd9d4a2a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.293481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1290/2599] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/users/eff856b09e5944a9bdcf7374cd9d4a2a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.515591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.525529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee360598-c038-4b88-bf0a-70e20d0fff17 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.602020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.602835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.603100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.603455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] PATH_INFO: `/v3/projects/7d4531a73035420fb3f73e1fd43af258` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.604956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d4531a73035420fb3f73e1fd43af258)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.607391 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.629718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.632488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.664446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d715d0-ad1b-4d0a-bd64-4c8edefbb3a9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.669832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d4531a73035420fb3f73e1fd43af258', 'request_id': 'req-bcebc370-a8a6-415b-a939-4054d33ed39c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.677995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d4531a73035420fb3f73e1fd43af258', 'request_id': 'req-bcebc370-a8a6-415b-a939-4054d33ed39c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.682319 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:34.689126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:34.689293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d4531a73035420fb3f73e1fd43af258', 'request_id': 'req-bcebc370-a8a6-415b-a939-4054d33ed39c'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.709202 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] The token cache is being invalidate because project 7d4531a73035420fb3f73e1fd43af258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:34.709538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.710420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bcebc370-a8a6-415b-a939-4054d33ed39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.712137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1310/2600] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/projects/7d4531a73035420fb3f73e1fd43af258 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.719175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f2dd9-31d5-4c43-b604-d6fa5182d978 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.719461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f2dd9-31d5-4c43-b604-d6fa5182d978 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.721675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0f2dd9-31d5-4c43-b604-d6fa5182d978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.739159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.739824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.740168 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.740565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] PATH_INFO: `/v3/projects/0d4afcdd7ba94aba95af66c6ebc8be94` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.740818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-bc0f2dd9-31d5-4c43-b604-d6fa5182d978 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:34.742094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d4afcdd7ba94aba95af66c6ebc8be94)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.742386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bc0f2dd9-31d5-4c43-b604-d6fa5182d978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.744336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.752514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1311/2601] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:34.761720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.762082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.762361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.786733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:34.787983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.803878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d4afcdd7ba94aba95af66c6ebc8be94', 'request_id': 'req-742d920b-5260-4503-9b2c-e2d279aa6ed8'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.811999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d4afcdd7ba94aba95af66c6ebc8be94', 'request_id': 'req-742d920b-5260-4503-9b2c-e2d279aa6ed8'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.821583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d4afcdd7ba94aba95af66c6ebc8be94', 'request_id': 'req-742d920b-5260-4503-9b2c-e2d279aa6ed8'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.846665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] The token cache is being invalidate because project 0d4afcdd7ba94aba95af66c6ebc8be94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:34.846989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.847734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:34.849294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1291/2602] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/projects/0d4afcdd7ba94aba95af66c6ebc8be94 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:34.853380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.855015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:34.869273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1312/2603] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:34.870566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.881174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94372a8-29ef-4560-bbce-a279ddc76674 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.886102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.886836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.887213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.887614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.888366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:34.888765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.890571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.909507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1313/2604] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:34.912716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:34.926342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b35b3-19b3-4e0e-b578-b6bcfb6a47c1 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.932252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.932871 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.933168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.933415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.934095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.936575 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.945980 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742d920b-5260-4503-9b2c-e2d279aa6ed8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:34.951925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:34.952763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:34.953110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:34.953452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] PATH_INFO: `/v3/users/3422881e81344497a286a44e8af74d52` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:34.954927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3422881e81344497a286a44e8af74d52)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:34.957421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:34.970510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1314/2605] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:34 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:34.971735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.035763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3422881e81344497a286a44e8af74d52'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.047262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cdaab-031f-4d7b-b426-671c6e4968c7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.048309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3422881e81344497a286a44e8af74d52'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.050088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3422881e81344497a286a44e8af74d52'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.051528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3422881e81344497a286a44e8af74d52'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.052743 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.053496 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.053761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:35.053886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.054546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.054819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.056744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.075408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3422881e81344497a286a44e8af74d52'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.084214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1315/2606] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:34 2022] POST /identity/v3/users => generated 285 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:35.085071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.087440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.102217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981dba96-5b63-4d85-84b3-186bc84809a4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.108872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.109483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.109791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.110071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.110887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.112887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.136432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1316/2607] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:35.136432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.139659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bcdad0-f09b-4b4a-9ecb-a89bd729d87e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.144203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1292/2608] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:34 2022] DELETE /identity/v3/users/3422881e81344497a286a44e8af74d52 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:35.178124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:35.231806 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487fe8a-bf46-4290-be60-c77680d6280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.232760 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487fe8a-bf46-4290-be60-c77680d6280c admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.233683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487fe8a-bf46-4290-be60-c77680d6280c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.234187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487fe8a-bf46-4290-be60-c77680d6280c admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.235421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487fe8a-bf46-4290-be60-c77680d6280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.237770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487fe8a-bf46-4290-be60-c77680d6280c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.256894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1317/2609] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:35.263680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.284524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac834ceb-4153-4d62-bcd9-2383601cf1a8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.299457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.300378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.304228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.304666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] PATH_INFO: `/v3/projects/76a9193a55cc4c249535afc5dc23aede/users/c1598c39701248d28e886f97224b85b2/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.331442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76a9193a55cc4c249535afc5dc23aede, user_id=c1598c39701248d28e886f97224b85b2, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.335713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.377551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1293/2610] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:35 2022] PUT /identity/v3/projects/76a9193a55cc4c249535afc5dc23aede/users/c1598c39701248d28e886f97224b85b2/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:35.389588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.390012 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.390382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.446190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] MFA Rules not processed for user `c1598c39701248d28e886f97224b85b2`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:35.447614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.532480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.534841 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.564401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1318/2611] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:35.577872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.658036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a2eef6-ae06-4e74-ba93-8cd3ae3fd664 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.671085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.671826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.673070 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af4adbf-d639-4cbd-b435-56e028afaa3e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.678164 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.678732 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] PATH_INFO: `/v3/projects/38687cd8068d4d02a514442609e67003` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.679519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.680352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.680526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38687cd8068d4d02a514442609e67003)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.680917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.681332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.682205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.682964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.684534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.723043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1319/2612] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:35.730817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.780335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:35.788047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38687cd8068d4d02a514442609e67003', 'request_id': 'req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.798733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38687cd8068d4d02a514442609e67003', 'request_id': 'req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.817706 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38687cd8068d4d02a514442609e67003', 'request_id': 'req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.840462 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b61d8f-b0a4-4155-809f-13670b5c59ad admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.864274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] The token cache is being invalidate because project 38687cd8068d4d02a514442609e67003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:35.864493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.866038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:35.870340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1294/2613] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:35 2022] DELETE /identity/v3/projects/38687cd8068d4d02a514442609e67003 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:35.874071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.878153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.925887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.926789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.927162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.927678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.928394 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.930508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.935709 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:35.959939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1320/2614] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:35 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:35.961331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:35.963284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:35.966032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5a0de5-3b32-47e3-863e-82d3d29d19a0 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.972008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.972859 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.973294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.973768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] PATH_INFO: `/v3/users/564ae28615a14b14a4be75f5d9d04713` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.975286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] RBAC: Authorizing `identity:delete_user(user_id=564ae28615a14b14a4be75f5d9d04713)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.977419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:35.978476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b16925-34f9-48e8-b6c6-e1d47a0251fa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:35.983925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:35.984640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:35.984982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:35.985339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:35.986085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:35.987737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.003137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1321/2615] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:36.008625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.029565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ddc04-e173-4aca-91ad-353a133d7b66 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.037176 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.038086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.038494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.038617 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] PATH_INFO: `/v3/projects/a015a556aa7f4bb3983ad0dc60a4ca8b/users/9571f166f73245419dfab21e6c8c67b7/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.055305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a015a556aa7f4bb3983ad0dc60a4ca8b, user_id=9571f166f73245419dfab21e6c8c67b7, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.056757 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '564ae28615a14b14a4be75f5d9d04713'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.058049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.066107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '564ae28615a14b14a4be75f5d9d04713'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.066843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '564ae28615a14b14a4be75f5d9d04713'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.067524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '564ae28615a14b14a4be75f5d9d04713'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.075218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '564ae28615a14b14a4be75f5d9d04713'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.075767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1322/2616] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:36 2022] PUT /identity/v3/projects/a015a556aa7f4bb3983ad0dc60a4ca8b/users/9571f166f73245419dfab21e6c8c67b7/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:36.079860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.083777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.101811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1295/2617] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:35 2022] DELETE /identity/v3/users/564ae28615a14b14a4be75f5d9d04713 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:36.155223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478e1eb-326d-4506-955c-bdb7e3684b49 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.205417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:36.223746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5c61bf-78c2-46e1-a91b-6f6432f0b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.224497 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c61bf-78c2-46e1-a91b-6f6432f0b6ce admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.224852 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c61bf-78c2-46e1-a91b-6f6432f0b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.225181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5c61bf-78c2-46e1-a91b-6f6432f0b6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.226858 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c61bf-78c2-46e1-a91b-6f6432f0b6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.228217 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5c61bf-78c2-46e1-a91b-6f6432f0b6ce admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.242371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1323/2618] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/roles => generated 1420 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:36.243610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.263810 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1353376-7f09-4f3d-935e-b06d8d2c25df admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.269797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.270622 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.270952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.271321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] PATH_INFO: `/v3/projects/a015a556aa7f4bb3983ad0dc60a4ca8b/users/9571f166f73245419dfab21e6c8c67b7/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.273722 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a015a556aa7f4bb3983ad0dc60a4ca8b, user_id=9571f166f73245419dfab21e6c8c67b7, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.276331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.292088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1296/2619] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:36 2022] PUT /identity/v3/projects/a015a556aa7f4bb3983ad0dc60a4ca8b/users/9571f166f73245419dfab21e6c8c67b7/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:36.302964 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.303256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.303535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.336937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] MFA Rules not processed for user `9571f166f73245419dfab21e6c8c67b7`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:36.338078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.459797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1324/2620] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:36.463246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.465388 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.468875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.471779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.571495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b61d1b7-880b-42fc-aa38-3236a32dae44 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.574841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd31c-16dc-44dd-a656-800e7b9d918c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.576287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.577054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.577340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.577614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] PATH_INFO: `/v3/projects/16e295732a1146b9a62bb6787db4bb00` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.578919 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=16e295732a1146b9a62bb6787db4bb00)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.580513 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60120deb-d861-4ff6-9ddb-143c3b56ceaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.580979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.581509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120deb-d861-4ff6-9ddb-143c3b56ceaa service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.582001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120deb-d861-4ff6-9ddb-143c3b56ceaa service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.582598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60120deb-d861-4ff6-9ddb-143c3b56ceaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.599109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120deb-d861-4ff6-9ddb-143c3b56ceaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.601352 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60120deb-d861-4ff6-9ddb-143c3b56ceaa service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.623696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1325/2621] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:36.643449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:36.647408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16e295732a1146b9a62bb6787db4bb00', 'request_id': 'req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.665874 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16e295732a1146b9a62bb6787db4bb00', 'request_id': 'req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.695252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16e295732a1146b9a62bb6787db4bb00', 'request_id': 'req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.718794 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] The token cache is being invalidate because project 16e295732a1146b9a62bb6787db4bb00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:36.719153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.719930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9601405a-cefe-4d5e-9b4f-8d69deb5fc6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:36.725148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1297/2622] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:36 2022] DELETE /identity/v3/projects/16e295732a1146b9a62bb6787db4bb00 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:36.726775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.727236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.727408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.759962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:36.791989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.898841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1326/2623] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:36.901462 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.902057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.902677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.933981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:36.939064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:36.969411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:36.970065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1298/2624] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:36.979655 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a5500-9d5e-4cec-b2ef-7dc0045475c4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:36.986133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:36.986832 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:36.987119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:36.987409 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:36.988142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:36.988475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:36.990295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:36.993095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:37.022393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1327/2625] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:26:37.023583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.039625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ea8b14-0419-43f9-a2ad-873658896332 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.044990 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.045590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.045944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.046225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.046902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.048958 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.099612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1299/2626] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:37 2022] POST /identity/v3/projects => generated 363 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:37.103697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.124909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d28f17-a317-48a1-9192-dd966fa6d207 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.269416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.270134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.270416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.270686 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.271376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.274398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.300361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1328/2627] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:37 2022] POST /identity/v3/users => generated 282 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:37.304440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.329814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40949cca-244f-4cf8-86a0-4b3fc52c91a1 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.335887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.336262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.336533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.336800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.337451 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.339508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.350090 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1300/2628] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:37.354442 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.366051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1109e919-da75-4ee9-b11a-c973018f58ca admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.371264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d2114-814d-4af0-9825-d8340196a87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.371872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d2114-814d-4af0-9825-d8340196a87b admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.372150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d2114-814d-4af0-9825-d8340196a87b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.372543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d2114-814d-4af0-9825-d8340196a87b admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.373196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d2114-814d-4af0-9825-d8340196a87b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.375378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4d2114-814d-4af0-9825-d8340196a87b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.395953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1329/2629] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:37.397066 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.407258 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe9b6b-b645-4e63-809d-860fcf563ebf admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.412094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.412752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.413049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.413321 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] PATH_INFO: `/v3/projects/4d5bef441a7b45af8f35acc2a91d5d3f/users/0dda7042b686482eb27324021cb8a811/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.428935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d5bef441a7b45af8f35acc2a91d5d3f, user_id=0dda7042b686482eb27324021cb8a811, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.431719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.451362 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1301/2630] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:37 2022] PUT /identity/v3/projects/4d5bef441a7b45af8f35acc2a91d5d3f/users/0dda7042b686482eb27324021cb8a811/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:37.464292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.464604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.465039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.494766 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] MFA Rules not processed for user `0dda7042b686482eb27324021cb8a811`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:37.495990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:37.576580 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1330/2631] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:37.590208 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:37.593550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:37.651247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18259895-3688-44b0-827a-eef89a8fcb44 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:37.654945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2b21c-fc0a-422a-90e9-4a9b70cef63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:37.655560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2b21c-fc0a-422a-90e9-4a9b70cef63e service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:37.655881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2b21c-fc0a-422a-90e9-4a9b70cef63e service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:37.656094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2b21c-fc0a-422a-90e9-4a9b70cef63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:37.665638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2b21c-fc0a-422a-90e9-4a9b70cef63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:37.666368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2b21c-fc0a-422a-90e9-4a9b70cef63e service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:37.675712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1302/2632] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:38.335393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.343427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.416131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6fcb5-680a-4c47-a5b3-cbd353cef61b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.421033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.421748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.422086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.422475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] PATH_INFO: `/v3/users/c1598c39701248d28e886f97224b85b2` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.423651 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1598c39701248d28e886f97224b85b2)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.425712 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.431195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:38.481342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1598c39701248d28e886f97224b85b2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.490296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1598c39701248d28e886f97224b85b2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.490933 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1598c39701248d28e886f97224b85b2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.491535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1598c39701248d28e886f97224b85b2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.499729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1598c39701248d28e886f97224b85b2'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:38.519808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1331/2633] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:38 2022] DELETE /identity/v3/users/c1598c39701248d28e886f97224b85b2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:38.524081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.524359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.524653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.545110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:38.546335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.610797 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1303/2634] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:38.620227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:38.622144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:38.700657 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21fa5d0-95ce-44b3-b115-5f92f44b862b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:38.708560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:38.709327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:38.709650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:38.710029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:38.722526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:38.723963 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:38.741031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1332/2635] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:39.019212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:39.021048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:39.039093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca663eb-b54e-4559-b789-de7f55916c73 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:39.057028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:39.057716 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:39.058016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:39.058278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] PATH_INFO: `/v3/projects/76a9193a55cc4c249535afc5dc23aede` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:39.060063 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76a9193a55cc4c249535afc5dc23aede)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:39.062834 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:39.074637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:39.124442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76a9193a55cc4c249535afc5dc23aede', 'request_id': 'req-a8605786-f481-4c85-ad92-4fc8d5a94d3a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.132033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76a9193a55cc4c249535afc5dc23aede', 'request_id': 'req-a8605786-f481-4c85-ad92-4fc8d5a94d3a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.156159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76a9193a55cc4c249535afc5dc23aede', 'request_id': 'req-a8605786-f481-4c85-ad92-4fc8d5a94d3a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.169994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] The token cache is being invalidate because project 76a9193a55cc4c249535afc5dc23aede was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:39.170823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.171733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.176621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1304/2636] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:39 2022] DELETE /identity/v3/projects/76a9193a55cc4c249535afc5dc23aede => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:39.178856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:39.192653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179cb3a-02b0-4bcb-8f8a-5b200e56620f service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:39.277717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:39.278425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:39.278704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:39.278976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] PATH_INFO: `/v3/users/9571f166f73245419dfab21e6c8c67b7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:39.280186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9571f166f73245419dfab21e6c8c67b7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:39.282132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:39.318039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:39.344853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9571f166f73245419dfab21e6c8c67b7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.355094 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9571f166f73245419dfab21e6c8c67b7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.355750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9571f166f73245419dfab21e6c8c67b7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.356406 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9571f166f73245419dfab21e6c8c67b7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.364605 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-31d9538e-a161-4481-8394-274f7de09d57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9571f166f73245419dfab21e6c8c67b7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.386255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1333/2637] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:39 2022] DELETE /identity/v3/users/9571f166f73245419dfab21e6c8c67b7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:39.714386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:39.716465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:39.734952 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8605786-f481-4c85-ad92-4fc8d5a94d3a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:39.823872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:39.826224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:39.826224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:39.826224 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] PATH_INFO: `/v3/projects/a015a556aa7f4bb3983ad0dc60a4ca8b` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:39.830535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a015a556aa7f4bb3983ad0dc60a4ca8b)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:39.835103 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:39.890416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a015a556aa7f4bb3983ad0dc60a4ca8b', 'request_id': 'req-5ce50966-dc6e-470c-959c-e737f5f91bc3'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.897257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a015a556aa7f4bb3983ad0dc60a4ca8b', 'request_id': 'req-5ce50966-dc6e-470c-959c-e737f5f91bc3'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.906254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a015a556aa7f4bb3983ad0dc60a4ca8b', 'request_id': 'req-5ce50966-dc6e-470c-959c-e737f5f91bc3'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.919250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] The token cache is being invalidate because project a015a556aa7f4bb3983ad0dc60a4ca8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:39.919539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.920217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-5ce50966-dc6e-470c-959c-e737f5f91bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:39.921982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1305/2638] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:39 2022] DELETE /identity/v3/projects/a015a556aa7f4bb3983ad0dc60a4ca8b => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:39.926827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:39.927231 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:39.927726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:39.950825 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:39.952710 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:40.031338 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1334/2639] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:40.039561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.040072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.040072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.060231 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:40.061469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:40.069615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1306/2640] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:40.073847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.084275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960b491-5f5d-4ebc-b56b-add55dc9addc None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.088906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.089573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.089945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.090293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.091059 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:40.091360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.093615 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.107558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1335/2641] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:40.112353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.124916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1620432-3618-4e2a-8f7e-4c3cb9497c43 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.129828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.130428 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.130700 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.130961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.131620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.133634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.156469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:40.166876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1307/2642] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:40.167980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.178334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1368b4-baba-44bb-9fb8-fc82534ae21f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.234861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.235472 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.235751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.236033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.236762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.238733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.263883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1336/2643] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:40.265891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.276042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65115ed5-8b1c-47d2-bbbf-c2ff38624380 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.280591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.281173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.281443 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.281708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.282375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.284218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.294078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1308/2644] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:40.297841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.308733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec97be-4997-4249-b9bc-f9242a2f0388 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.313126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95619ca0-7809-4a70-abc3-8f529d8363b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.313721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95619ca0-7809-4a70-abc3-8f529d8363b0 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.314030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95619ca0-7809-4a70-abc3-8f529d8363b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.314322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95619ca0-7809-4a70-abc3-8f529d8363b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.314982 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95619ca0-7809-4a70-abc3-8f529d8363b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.316574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95619ca0-7809-4a70-abc3-8f529d8363b0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.330473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1337/2645] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:40.332410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.342656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2127a9-304f-4b8f-bdfa-67bc50b19bd8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.347111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.347747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.348024 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.348294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] PATH_INFO: `/v3/projects/726162ed88e447298b79c400e126bdb4/users/771a0b18ac924e7d8629c4f39b972540/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.362827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726162ed88e447298b79c400e126bdb4, user_id=771a0b18ac924e7d8629c4f39b972540, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.365432 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.381306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1309/2646] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:40 2022] PUT /identity/v3/projects/726162ed88e447298b79c400e126bdb4/users/771a0b18ac924e7d8629c4f39b972540/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:40.392131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.392584 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.392851 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.416659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:40.425296 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] MFA Rules not processed for user `771a0b18ac924e7d8629c4f39b972540`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:40.427493 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:40.491448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1338/2647] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:40.495931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.560454 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6a4de7-5df2-4130-b005-1f3d25eab015 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.565461 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.566082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.566357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.566624 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.567241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.569101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.600269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1310/2648] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:40.602267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.613670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db151b-503d-41ad-8cf3-7c580c970c2c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.672377 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.672992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.673286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.673587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.674262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.676284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.710011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1339/2649] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/users => generated 303 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:40.711292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.724715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192907c-6a52-4082-a9d7-52dc4c428837 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.729528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.730281 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.730561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.730828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.731470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.733646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.744995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1311/2650] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:40.750117 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.759995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc27f43-9dea-4aab-99b0-fd90f6089c35 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.764620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.765262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.765537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.765836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] PATH_INFO: `/v3/projects/2eb6528cb5d4443cac0e06197e2139e2/users/1f806f5ff9f545f9a7612a13298c0b57/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.792292 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb6528cb5d4443cac0e06197e2139e2, user_id=1f806f5ff9f545f9a7612a13298c0b57, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.795792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.815421 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1340/2651] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:40 2022] PUT /identity/v3/projects/2eb6528cb5d4443cac0e06197e2139e2/users/1f806f5ff9f545f9a7612a13298c0b57/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:40.819866 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.831623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e31a5f3-5de0-44ce-961f-9d93a90b0310 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.921505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e20350-715c-46e0-bee2-d6f2e7b0d539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.922578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20350-715c-46e0-bee2-d6f2e7b0d539 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.923157 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20350-715c-46e0-bee2-d6f2e7b0d539 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.923663 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e20350-715c-46e0-bee2-d6f2e7b0d539 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.924658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e20350-715c-46e0-bee2-d6f2e7b0d539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.928897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e20350-715c-46e0-bee2-d6f2e7b0d539 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.940269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1312/2652] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:40.944544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:40.961935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19d9e6-10bd-4e36-878d-571ac9eda220 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:40.964174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:40.964996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.965274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.965545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] PATH_INFO: `/v3/projects/2eb6528cb5d4443cac0e06197e2139e2/users/1f806f5ff9f545f9a7612a13298c0b57/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:40.967986 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2eb6528cb5d4443cac0e06197e2139e2, user_id=1f806f5ff9f545f9a7612a13298c0b57, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:40.970761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:40.981610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1341/2653] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:40 2022] PUT /identity/v3/projects/2eb6528cb5d4443cac0e06197e2139e2/users/1f806f5ff9f545f9a7612a13298c0b57/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:40.993614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:40.993945 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:40.994218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:41.027320 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] MFA Rules not processed for user `1f806f5ff9f545f9a7612a13298c0b57`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:41.028584 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:41.053617 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:41.113311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:41.115155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:41.123204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1313/2654] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:41.134951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:41.136283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:41.177000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b626e24-aece-478f-8482-b5f34bd507cc admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:41.180674 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:41.181280 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:41.181558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:41.181875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:41.183647 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:41.189718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:41.192606 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dd9549-704a-44fa-b143-893afc914bbc None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:41.196529 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:41.197255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:41.197551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:41.197964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:41.207710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:41.209122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:41.220282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1314/2655] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:41 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:26:41.253432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:41.254920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:41.265276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1342/2656] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:41.268236 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:41.269792 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:41.285514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd32f91-29bc-447e-8f03-9d8f5f441b4c service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:41.289287 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:41.290096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:41.290437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:41.290941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:41.350408 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:41.351917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:41.362128 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1315/2657] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:44.626876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:44.628491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:44.642352 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:44.643454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f43ca08-c10c-48c5-8e51-0cb10f1c1375 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:44.644457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:44.648720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:44.649346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:44.649626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:44.649929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] PATH_INFO: `/v3/users/2569b0dabcc146bba73908b14427a0bb` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:44.651207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2569b0dabcc146bba73908b14427a0bb)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:44.653235 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:44.662738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ba28-8093-45cb-a77e-bd6d67b71040 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:44.669518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:44.670674 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:44.670944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:44.671237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] PATH_INFO: `/v3/users/91a5130420764ac78383961d7665a964` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:44.672629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91a5130420764ac78383961d7665a964)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:44.676632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:44.716218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2569b0dabcc146bba73908b14427a0bb'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.727005 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2569b0dabcc146bba73908b14427a0bb'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.727763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2569b0dabcc146bba73908b14427a0bb'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.728522 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2569b0dabcc146bba73908b14427a0bb'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.740276 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-dd9e3a3f-35fd-4735-bad7-640c1aa526d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2569b0dabcc146bba73908b14427a0bb'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.751142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91a5130420764ac78383961d7665a964'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.761963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91a5130420764ac78383961d7665a964'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.765096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1343/2658] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:44 2022] DELETE /identity/v3/users/2569b0dabcc146bba73908b14427a0bb => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:44.765690 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91a5130420764ac78383961d7665a964'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.766415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91a5130420764ac78383961d7665a964'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.767878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:44.770640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:44.771079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:44.771390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:44.782289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91a5130420764ac78383961d7665a964'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:44.795935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:44.797204 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:44.807130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1316/2659] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:44 2022] DELETE /identity/v3/users/91a5130420764ac78383961d7665a964 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:44.885960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1344/2660] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:44.897955 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:44.899485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.001456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.003385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.004276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9bdb48-3ca0-4e36-9be5-a867019bfe32 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.008394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.009129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.009499 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.009900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.020216 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988419ed-6768-4fdb-ba83-bfaa2a01c75d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.039736 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:45.089936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.091379 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.098764 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.099400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.099680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.099953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] PATH_INFO: `/v3/users/771a0b18ac924e7d8629c4f39b972540` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.101130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=771a0b18ac924e7d8629c4f39b972540)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.101897 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1317/2661] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:45.102693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.103155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.104034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.116719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712642a-74ad-4780-83a0-bb4b0c1e6d66 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.121871 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.122407 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.122679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.123042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] PATH_INFO: `/v3/projects/b6912d8a0f544d9d9b2d9b1d1048952d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.124893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6912d8a0f544d9d9b2d9b1d1048952d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.127611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.161273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '771a0b18ac924e7d8629c4f39b972540'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.171244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '771a0b18ac924e7d8629c4f39b972540'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.172347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '771a0b18ac924e7d8629c4f39b972540'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.173058 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '771a0b18ac924e7d8629c4f39b972540'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.183935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-7e6681b8-52d3-4d23-8912-5ddc22b9da45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '771a0b18ac924e7d8629c4f39b972540'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.192691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6912d8a0f544d9d9b2d9b1d1048952d', 'request_id': 'req-b50ebd84-b004-40e2-9752-c935714e44a1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.199861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6912d8a0f544d9d9b2d9b1d1048952d', 'request_id': 'req-b50ebd84-b004-40e2-9752-c935714e44a1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.208632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1345/2662] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:45 2022] DELETE /identity/v3/users/771a0b18ac924e7d8629c4f39b972540 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:45.210261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6912d8a0f544d9d9b2d9b1d1048952d', 'request_id': 'req-b50ebd84-b004-40e2-9752-c935714e44a1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.213511 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.213920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.214203 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.223313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] The token cache is being invalidate because project b6912d8a0f544d9d9b2d9b1d1048952d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:45.223607 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.224603 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b50ebd84-b004-40e2-9752-c935714e44a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.226399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1318/2663] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:45 2022] DELETE /identity/v3/projects/b6912d8a0f544d9d9b2d9b1d1048952d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:45.232269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ff311-f0b0-44b4-8df2-70ede737d5a3 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.232634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ff311-f0b0-44b4-8df2-70ede737d5a3 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.232921 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ff311-f0b0-44b4-8df2-70ede737d5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.237346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:45.238698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.246951 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:45.259402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-034ff311-f0b0-44b4-8df2-70ede737d5a3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:45.261126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-034ff311-f0b0-44b4-8df2-70ede737d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.313432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1346/2664] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:45.320796 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.323074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.333512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1319/2665] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:45.341097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.341378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.341649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.364556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:45.365977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.371162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:45.379027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1320/2666] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:45.380206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.393317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f2858d-6d64-4058-ae35-208e0354d27d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.396388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ed994-96eb-43ba-9edb-d228de3a00b4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.398890 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.399608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.400086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.400361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.400466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.401087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.401205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:45.401341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.401589 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.401837 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.403665 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.415541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.415541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.421771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1321/2667] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:45.422697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.424481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.429961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1347/2668] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:45.430714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.442547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a29276-832f-4d79-9f23-8cc734faa6d4 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.444915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524935f-180d-4a7a-926c-eb77ec1761b1 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.448397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.449190 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.449683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.450067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.450949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.451534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.452177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.452433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.452705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] PATH_INFO: `/v3/projects/cd39a0a036eb47cabfdd1840a9355322` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.453939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.454053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd39a0a036eb47cabfdd1840a9355322)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.455918 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.490913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1348/2669] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:45.495376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.511448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd39a0a036eb47cabfdd1840a9355322', 'request_id': 'req-54468f1c-5075-4d17-bba5-fde66ad312d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.520710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd39a0a036eb47cabfdd1840a9355322', 'request_id': 'req-54468f1c-5075-4d17-bba5-fde66ad312d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.531448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd39a0a036eb47cabfdd1840a9355322', 'request_id': 'req-54468f1c-5075-4d17-bba5-fde66ad312d7'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.546134 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] The token cache is being invalidate because project cd39a0a036eb47cabfdd1840a9355322 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:45.546466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.547404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-54468f1c-5075-4d17-bba5-fde66ad312d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.549114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1322/2670] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:45 2022] DELETE /identity/v3/projects/cd39a0a036eb47cabfdd1840a9355322 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:45.554007 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c93f-b475-4205-a562-ab91fb5a837c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.554284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c93f-b475-4205-a562-ab91fb5a837c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.554615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f26c93f-b475-4205-a562-ab91fb5a837c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.576729 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-5f26c93f-b475-4205-a562-ab91fb5a837c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:45.577245 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e98c5a-ab4b-49d3-845d-1f4e9a4059c7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.578015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5f26c93f-b475-4205-a562-ab91fb5a837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.642633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.643456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.643940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.644076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.644815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.646830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.656571 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1323/2671] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:45.666317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.666616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.666947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.673785 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1349/2672] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:45.676931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.678653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.690288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8513-2aae-4521-8988-b43c0fe7edc6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.690946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:45.692149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.695258 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.695855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.696128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.696398 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.697031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.698628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.700325 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1324/2673] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:45.705273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.709895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:45.710506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1350/2674] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:45.714559 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.719593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43222ba5-06b9-429f-8a37-686c2131b635 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.725770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2abdc-f46f-468f-8e8e-abbb3bab7dba admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.727225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.729077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.729278 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.729572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.730821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:45.731417 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.734264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.734264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.734264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.734264 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.734623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.734729 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.736049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.747889 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1351/2675] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:45.748807 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1325/2676] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:26:45.753471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.754211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.764995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044568c8-0fde-4f3d-acff-0d2ad61434a8 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.766178 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4a3c-8165-4273-95d5-c6dbc468e30b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.772186 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.773284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.773817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.773944 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.774845 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] PATH_INFO: `/v3/projects/820af9ba18b347118ab7c6168abf7456/users/fc82a57b2dba4fe8a6f5c41045f38b8d/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.774994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.774994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.774994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.775743 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.777787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.796423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=820af9ba18b347118ab7c6168abf7456, user_id=fc82a57b2dba4fe8a6f5c41045f38b8d, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.800332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.810120 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1326/2677] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:45.811003 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.814242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.814945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1352/2678] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:45 2022] PUT /identity/v3/projects/820af9ba18b347118ab7c6168abf7456/users/fc82a57b2dba4fe8a6f5c41045f38b8d/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:45.823436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.831159 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:45.888215 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76d170-9dd5-4263-851d-8009cd0d6a87 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.893601 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a92978-8cf6-48f9-b03a-9a41c6b88f9d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:45.893784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.894255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.894523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.894789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] PATH_INFO: `/v3/projects/726162ed88e447298b79c400e126bdb4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.895893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] RBAC: Authorizing `identity:delete_project(project_id=726162ed88e447298b79c400e126bdb4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.897799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.898349 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc8bc2-26ce-4847-b865-33626be19d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:45.898937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc8bc2-26ce-4847-b865-33626be19d3d admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.899212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc8bc2-26ce-4847-b865-33626be19d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.899508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc8bc2-26ce-4847-b865-33626be19d3d admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.900126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc8bc2-26ce-4847-b865-33626be19d3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:45.902025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc8bc2-26ce-4847-b865-33626be19d3d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:45.929335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1353/2679] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:45.931372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:45.931672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:45.931928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:45.949209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726162ed88e447298b79c400e126bdb4', 'request_id': 'req-b8e79f68-eba7-4685-b807-288ccd79b28a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.955829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726162ed88e447298b79c400e126bdb4', 'request_id': 'req-b8e79f68-eba7-4685-b807-288ccd79b28a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.961433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] MFA Rules not processed for user `fc82a57b2dba4fe8a6f5c41045f38b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:45.962946 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:45.963655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726162ed88e447298b79c400e126bdb4', 'request_id': 'req-b8e79f68-eba7-4685-b807-288ccd79b28a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.975509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] The token cache is being invalidate because project 726162ed88e447298b79c400e126bdb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:45.975905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.976548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:45.978074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1327/2680] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:45 2022] DELETE /identity/v3/projects/726162ed88e447298b79c400e126bdb4 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:45.978823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:45.980106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.041144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1354/2681] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:46.042745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:46.046263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.047608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.048547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e79f68-eba7-4685-b807-288ccd79b28a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.054594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.055307 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.055592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.056114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.057033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.058806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.062218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa2320-04b4-4e15-b1d5-2e957a7b5c33 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.066949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.067571 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.067853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.068122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] PATH_INFO: `/v3/users/1f806f5ff9f545f9a7612a13298c0b57` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.068933 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1328/2682] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:46.069240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f806f5ff9f545f9a7612a13298c0b57)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.069704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.071177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.071322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.121539 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f806f5ff9f545f9a7612a13298c0b57'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.129423 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f806f5ff9f545f9a7612a13298c0b57'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.130223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f806f5ff9f545f9a7612a13298c0b57'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.130948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f806f5ff9f545f9a7612a13298c0b57'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.139347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f806f5ff9f545f9a7612a13298c0b57'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.152556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:46.152731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e78a5e3-bda9-4d81-a9b2-d195741fb453 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.159588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.160509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.160783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.161051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.162607 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1355/2683] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:46 2022] DELETE /identity/v3/users/1f806f5ff9f545f9a7612a13298c0b57 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:46.163804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.165611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.166657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.184969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.187130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.197211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1329/2684] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:46.259966 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9600bb-78a1-4654-9587-0dba5ca61f10 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.265883 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d91ec8-2815-483f-8899-404a3b167dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.266479 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d91ec8-2815-483f-8899-404a3b167dc3 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.266870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d91ec8-2815-483f-8899-404a3b167dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.267225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d91ec8-2815-483f-8899-404a3b167dc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.267908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d91ec8-2815-483f-8899-404a3b167dc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.269935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d91ec8-2815-483f-8899-404a3b167dc3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.285486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.289975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1356/2685] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:46 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:46.296555 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677fa91a-dd43-4b75-a359-e267d0edae4a service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.308518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.309276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.309648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.310181 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] PATH_INFO: `/v3/projects/c9d02ad902bd4e50a7f65239e84e33c7/users/7b6f69d99c2342b4a93aa124d20d499d/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.335505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9d02ad902bd4e50a7f65239e84e33c7, user_id=7b6f69d99c2342b4a93aa124d20d499d, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.338728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.353475 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1330/2686] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:46 2022] PUT /identity/v3/projects/c9d02ad902bd4e50a7f65239e84e33c7/users/7b6f69d99c2342b4a93aa124d20d499d/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:46.364919 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.365229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.365471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.398675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] MFA Rules not processed for user `7b6f69d99c2342b4a93aa124d20d499d`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:46.400023 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.428596 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.430870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.484342 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1357/2687] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:46 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:46.485988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.487835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.526153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf1a2a-d097-4a3d-9da1-3031a01978b6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.531299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.531854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.532151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.532402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] PATH_INFO: `/v3/users/fc82a57b2dba4fe8a6f5c41045f38b8d` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.533847 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc82a57b2dba4fe8a6f5c41045f38b8d)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.535787 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.556275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd2bce6-d113-4c1a-a64c-bc27826113a6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.561519 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.562182 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.562464 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.562732 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] PATH_INFO: `/v3/projects/2eb6528cb5d4443cac0e06197e2139e2` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.563874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2eb6528cb5d4443cac0e06197e2139e2)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.565875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.582710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc82a57b2dba4fe8a6f5c41045f38b8d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.590848 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc82a57b2dba4fe8a6f5c41045f38b8d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.591628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc82a57b2dba4fe8a6f5c41045f38b8d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.592380 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc82a57b2dba4fe8a6f5c41045f38b8d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.594509 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:46.599075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:46.601421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc82a57b2dba4fe8a6f5c41045f38b8d'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.617395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2eb6528cb5d4443cac0e06197e2139e2', 'request_id': 'req-63e3b114-9f4a-4572-a220-1b6ebe057073'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.620400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1331/2688] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:46 2022] DELETE /identity/v3/users/fc82a57b2dba4fe8a6f5c41045f38b8d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:46.621419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.622842 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.626727 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2eb6528cb5d4443cac0e06197e2139e2', 'request_id': 'req-63e3b114-9f4a-4572-a220-1b6ebe057073'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.635492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2eb6528cb5d4443cac0e06197e2139e2', 'request_id': 'req-63e3b114-9f4a-4572-a220-1b6ebe057073'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.645931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] The token cache is being invalidate because project 2eb6528cb5d4443cac0e06197e2139e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:46.646507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.646928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-63e3b114-9f4a-4572-a220-1b6ebe057073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:46.648432 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1358/2689] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:46 2022] DELETE /identity/v3/projects/2eb6528cb5d4443cac0e06197e2139e2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:46.652913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576840-bdf8-46cb-9aa4-1801ba41c9a8 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.653198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576840-bdf8-46cb-9aa4-1801ba41c9a8 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.653473 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84576840-bdf8-46cb-9aa4-1801ba41c9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.673590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-84576840-bdf8-46cb-9aa4-1801ba41c9a8 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:46.674816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-84576840-bdf8-46cb-9aa4-1801ba41c9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.680289 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d716f5-21fb-46a8-af83-b3a8a6b94fa6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.683885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.684479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.684758 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.685016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.687302 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.734558 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1359/2690] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:46.736506 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057fde-c8c6-4909-8bfa-79a70c991c8d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.736793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057fde-c8c6-4909-8bfa-79a70c991c8d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.737066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01057fde-c8c6-4909-8bfa-79a70c991c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.751046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.752481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.760654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-01057fde-c8c6-4909-8bfa-79a70c991c8d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:46.761976 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-01057fde-c8c6-4909-8bfa-79a70c991c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.762200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1332/2691] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:46.763036 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.764631 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.773606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1360/2692] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:46.782126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.782669 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.783092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.809267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:46.811242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:46.819332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1361/2693] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:46.823646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.833289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4867ad9-9389-434f-b380-7f5db6efba65 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.838543 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.838925 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.839206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.839476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.840124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:46.840428 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.842089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.848204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc599a-6497-4c0f-883b-061d67e81294 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.853292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.853894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1362/2694] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:26:46.854008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.854172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.854424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.858810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.865201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.866769 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.868754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fe56d6-cd92-453d-a629-3d9ea1de5efa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.874025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:46.874553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:46.874827 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:46.875095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:46.875781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:46.877656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:46.878122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1333/2695] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:46.907503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1363/2696] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:46 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:46.912873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:46.924331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dbb3c-8936-417f-a93b-94f97589a6f8 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:46.964937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:47.001523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.002170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.002517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.002811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.003677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.005316 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.034218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1334/2697] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:46 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:47.038365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.054033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5a001c-2069-40f9-8702-cb0e73509b1f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.083030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.083658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.083936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.084242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.084935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.086695 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.102510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1364/2698] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:47.106592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.121040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd45204-9726-4d9c-996d-f6c974f5d59a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.130814 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.131502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.145770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.145770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.145770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.145770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.156759 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1335/2699] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:47.167723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.182091 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169ba8ef-a9e1-4400-82d1-c4c9feb4c8fe admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.187413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c86c44-c3a9-4447-81fa-cb1e1addaa74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.188183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86c44-c3a9-4447-81fa-cb1e1addaa74 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.188561 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86c44-c3a9-4447-81fa-cb1e1addaa74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.188928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c86c44-c3a9-4447-81fa-cb1e1addaa74 admin admin] PATH_INFO: `/v3/projects/c0ad02b2e16e4ca98e9d773005a177df/users/5cfbfc0235ca44f1a717d35f65d1e7f4/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.205978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c86c44-c3a9-4447-81fa-cb1e1addaa74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ad02b2e16e4ca98e9d773005a177df, user_id=5cfbfc0235ca44f1a717d35f65d1e7f4, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.209066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c86c44-c3a9-4447-81fa-cb1e1addaa74 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.224127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1365/2700] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:47 2022] PUT /identity/v3/projects/c0ad02b2e16e4ca98e9d773005a177df/users/5cfbfc0235ca44f1a717d35f65d1e7f4/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:47.226448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.228414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.234778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.235064 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.235337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.243601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffd231-d286-49df-be15-ff1e94d80707 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.267536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] MFA Rules not processed for user `5cfbfc0235ca44f1a717d35f65d1e7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:47.268914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.288754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:47.304938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.305565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.306936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.306936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] PATH_INFO: `/v3/projects/820af9ba18b347118ab7c6168abf7456` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.307578 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] RBAC: Authorizing `identity:delete_project(project_id=820af9ba18b347118ab7c6168abf7456)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.309446 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.338547 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1366/2701] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:47.343503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.353757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f7ce7d-6bc2-4d8f-b0bc-227cfe9b87d7 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.356940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '820af9ba18b347118ab7c6168abf7456', 'request_id': 'req-b5912b58-8630-403d-a7c3-428ed28a6aba'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.363760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '820af9ba18b347118ab7c6168abf7456', 'request_id': 'req-b5912b58-8630-403d-a7c3-428ed28a6aba'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.371456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '820af9ba18b347118ab7c6168abf7456', 'request_id': 'req-b5912b58-8630-403d-a7c3-428ed28a6aba'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.383009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] The token cache is being invalidate because project 820af9ba18b347118ab7c6168abf7456 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:47.383314 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.383995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-b5912b58-8630-403d-a7c3-428ed28a6aba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:47.385514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1336/2702] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:47 2022] DELETE /identity/v3/projects/820af9ba18b347118ab7c6168abf7456 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:47.390615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b22b73-84c3-4aef-b6a8-59c6d46cf6e5 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.390969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b22b73-84c3-4aef-b6a8-59c6d46cf6e5 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.391313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b22b73-84c3-4aef-b6a8-59c6d46cf6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.411065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-06b22b73-84c3-4aef-b6a8-59c6d46cf6e5 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:47.412263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-06b22b73-84c3-4aef-b6a8-59c6d46cf6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.413784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.414392 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.414670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.414945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.415564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.417451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.420119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1337/2703] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:47.429430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365d476-4308-4977-9759-e3798062be13 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.429899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365d476-4308-4977-9759-e3798062be13 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.430175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365d476-4308-4977-9759-e3798062be13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.444737 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1367/2704] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:47.448671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.450290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.451633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e365d476-4308-4977-9759-e3798062be13 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:47.452800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e365d476-4308-4977-9759-e3798062be13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.482001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:47.515271 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1338/2705] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:47.516653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7690b8-49c8-4239-a082-443b174ac48b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.519562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365d476-4308-4977-9759-e3798062be13 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.521418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.522046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.522326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.522597 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.523211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.525120 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.528200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365d476-4308-4977-9759-e3798062be13 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.533587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.534199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.534473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.535050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.535364 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:47.535981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.537219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.551034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1368/2706] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:47.551209 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1339/2707] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:26:47.552434 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.553719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.562717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729556a9-9f8d-423e-935e-12f0057e99f6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.563158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb255f3a-be42-4539-ac5c-2f087bd793e6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.567339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.567643 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.568043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.568210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.568327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.568483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.568582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.568747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.569221 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.569331 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.571037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.571223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.582745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1369/2708] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:47.586745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.598150 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a315-427e-44f4-b6f5-37d1140189f1 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.603418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1340/2709] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:47.604339 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.606325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.607157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.607480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.607762 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] PATH_INFO: `/v3/projects/fba7a415749a4e3f941341201fbe85a1/users/c60e2c8b6f8041f897dca94e49c30243/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.622595 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba7a415749a4e3f941341201fbe85a1, user_id=c60e2c8b6f8041f897dca94e49c30243, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.625821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.634816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:47.640424 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1370/2710] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:47 2022] PUT /identity/v3/projects/fba7a415749a4e3f941341201fbe85a1/users/c60e2c8b6f8041f897dca94e49c30243/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:47.644645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.666765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a031eda3-032e-4b85-abf0-59a831a47f11 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.712080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74099997-46ea-4bd0-86ae-1da59ca08d17 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.717244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.717874 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.718192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.718466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.719108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.720726 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.724485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.725083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.725361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.725633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.726286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.728161 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.730765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1371/2711] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:47.734939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.744665 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbcc6-da09-4ff7-a6f8-406752475460 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.749983 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efdcb8a-997c-4d37-8221-b6bd0d79a5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.750653 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdcb8a-997c-4d37-8221-b6bd0d79a5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.753703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1341/2712] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/users => generated 303 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:47.753843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdcb8a-997c-4d37-8221-b6bd0d79a5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.753843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efdcb8a-997c-4d37-8221-b6bd0d79a5a9 admin admin] PATH_INFO: `/v3/projects/fba7a415749a4e3f941341201fbe85a1/users/c60e2c8b6f8041f897dca94e49c30243/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.753843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdcb8a-997c-4d37-8221-b6bd0d79a5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba7a415749a4e3f941341201fbe85a1, user_id=c60e2c8b6f8041f897dca94e49c30243, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.754560 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.756236 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efdcb8a-997c-4d37-8221-b6bd0d79a5a9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.764430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079f180-386b-4f4d-943a-af6a981ddcaa admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.767282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1372/2713] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:47 2022] PUT /identity/v3/projects/fba7a415749a4e3f941341201fbe85a1/users/c60e2c8b6f8041f897dca94e49c30243/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:47.778463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.778758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.779037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.795985 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:47.807485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] MFA Rules not processed for user `c60e2c8b6f8041f897dca94e49c30243`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:47.808716 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.822802 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.823406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.823680 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.823947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.824587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.827647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.841784 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1342/2714] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:47.843965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.853632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd64469-4955-4673-8fa5-860a642e65b3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.858839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.859430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.859708 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.859979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.860644 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.862336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.872077 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1343/2715] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:47.876565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.882817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1373/2716] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:26:47.887013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beea533c-432c-4631-b64f-335dfbfe40a6 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.892123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41297ee-d0cd-4b37-a789-eea48d4d74ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:47.892832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41297ee-d0cd-4b37-a789-eea48d4d74ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.893239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41297ee-d0cd-4b37-a789-eea48d4d74ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.893518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41297ee-d0cd-4b37-a789-eea48d4d74ef admin admin] PATH_INFO: `/v3/projects/8890d0c9f4a345c8a4ab480facdfefa6/users/afe6026a633841fd8d9ad46dd69a2168/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.893938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:47.895306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.908610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41297ee-d0cd-4b37-a789-eea48d4d74ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=8890d0c9f4a345c8a4ab480facdfefa6, user_id=afe6026a633841fd8d9ad46dd69a2168, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:47.911257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41297ee-d0cd-4b37-a789-eea48d4d74ef admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:47.924886 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1344/2717] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:47 2022] PUT /identity/v3/projects/8890d0c9f4a345c8a4ab480facdfefa6/users/afe6026a633841fd8d9ad46dd69a2168/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:47.935145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:47.935994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:47.936329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:47.954123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca865256-b042-4aff-bebe-56efc0de4458 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:47.963474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] MFA Rules not processed for user `afe6026a633841fd8d9ad46dd69a2168`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:47.964742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:47.973253 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:48.003502 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.004109 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.004381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.004647 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.006405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:48.029713 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1345/2718] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:47 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:48.034280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.066370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.067814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.077529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1374/2719] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:48.096505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1710c838-49de-422a-a655-b15db8edf9a6 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.101021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.101623 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.101928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.102201 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.102831 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.104887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.135244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1346/2720] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:48.140872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.152893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd8260a-969a-4663-8ff6-ef8e761c9596 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.215819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.216489 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.216821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.217096 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.217887 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.219836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.245151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1375/2721] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:48.251016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.261936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a848e-301f-4d7b-b4a6-1e6fcd34c690 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.266974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.267688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.268035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.268315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.268963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.270642 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.276001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:48.282677 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1347/2722] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:48.285970 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.296719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477d6b23-7a42-4bc4-a009-6da3f62ac99b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.302006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.302658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.302935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.303232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] PATH_INFO: `/v3/projects/5cb681923cee45b4b072c841613df6ae/users/0f2bddf7c3974b289b5b609eb6d075c0/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.319957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb681923cee45b4b072c841613df6ae, user_id=0f2bddf7c3974b289b5b609eb6d075c0, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.323103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.337061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1376/2723] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:48 2022] PUT /identity/v3/projects/5cb681923cee45b4b072c841613df6ae/users/0f2bddf7c3974b289b5b609eb6d075c0/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:26:48.341405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.351931 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfcc6-693d-4df3-8183-b55958d50736 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.412664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43682a7e-1ddf-41c6-98a5-dede5f421a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.413353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43682a7e-1ddf-41c6-98a5-dede5f421a07 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.413696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43682a7e-1ddf-41c6-98a5-dede5f421a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.414053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43682a7e-1ddf-41c6-98a5-dede5f421a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.414703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43682a7e-1ddf-41c6-98a5-dede5f421a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.420684 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43682a7e-1ddf-41c6-98a5-dede5f421a07 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.435282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1348/2724] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:48.442181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.457067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55994d5-d8c8-414c-b315-82ba7984da53 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.465754 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.466429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.467240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.467564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] PATH_INFO: `/v3/projects/5cb681923cee45b4b072c841613df6ae/users/0f2bddf7c3974b289b5b609eb6d075c0/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.469837 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cb681923cee45b4b072c841613df6ae, user_id=0f2bddf7c3974b289b5b609eb6d075c0, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.472661 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.488134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1377/2725] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:48 2022] PUT /identity/v3/projects/5cb681923cee45b4b072c841613df6ae/users/0f2bddf7c3974b289b5b609eb6d075c0/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:48.498166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.498541 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.498889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.527014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] MFA Rules not processed for user `0f2bddf7c3974b289b5b609eb6d075c0`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:48.528216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:48.594482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1349/2726] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:48 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:48.605291 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:48.640302 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:48.666720 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65c65b8-913c-49fa-bdb7-4acf30a659ba admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:48.670908 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cc0e6-ba02-4ca9-a755-39a3de2ae775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:48.671509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cc0e6-ba02-4ca9-a755-39a3de2ae775 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:48.671786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cc0e6-ba02-4ca9-a755-39a3de2ae775 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:48.672052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cc0e6-ba02-4ca9-a755-39a3de2ae775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:48.680962 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cc0e6-ba02-4ca9-a755-39a3de2ae775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:48.682425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cc0e6-ba02-4ca9-a755-39a3de2ae775 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:48.691694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1378/2727] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:48 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:49.508909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.570761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce08f9-b5e9-477d-9248-7ed2d1e4865f None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.575206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.575825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.576657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.576929 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] PATH_INFO: `/v3/users/afe6026a633841fd8d9ad46dd69a2168` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.578071 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afe6026a633841fd8d9ad46dd69a2168)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.579966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.601363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:49.627194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afe6026a633841fd8d9ad46dd69a2168'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:49.634640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afe6026a633841fd8d9ad46dd69a2168'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:49.635251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afe6026a633841fd8d9ad46dd69a2168'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:49.635875 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afe6026a633841fd8d9ad46dd69a2168'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:49.642858 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afe6026a633841fd8d9ad46dd69a2168'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:49.660150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1350/2728] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:49 2022] DELETE /identity/v3/users/afe6026a633841fd8d9ad46dd69a2168 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:49.664583 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.665033 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.665294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.683996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:49.685191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:49.744667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1379/2729] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:49.754904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:49.756411 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:49.813139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a25d3-f790-4fde-8865-a6374664e946 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:49.816742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:49.817361 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:49.817638 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:49.817928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:49.826860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:49.828402 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:49.838253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1351/2730] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:50.094378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:50.095894 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:50.109056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abd645-967b-451d-a252-50db0e26f2bf None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:50.113888 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:50.114600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.114935 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.115273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] PATH_INFO: `/v3/projects/8890d0c9f4a345c8a4ab480facdfefa6` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:50.116453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8890d0c9f4a345c8a4ab480facdfefa6)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:50.118390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:50.166795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8890d0c9f4a345c8a4ab480facdfefa6', 'request_id': 'req-d6224cb9-0ac2-4345-8a55-a736de77e192'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.173401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8890d0c9f4a345c8a4ab480facdfefa6', 'request_id': 'req-d6224cb9-0ac2-4345-8a55-a736de77e192'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.180685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8890d0c9f4a345c8a4ab480facdfefa6', 'request_id': 'req-d6224cb9-0ac2-4345-8a55-a736de77e192'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.190763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] The token cache is being invalidate because project 8890d0c9f4a345c8a4ab480facdfefa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:50.191047 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.191668 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.193122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1380/2731] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:50 2022] DELETE /identity/v3/projects/8890d0c9f4a345c8a4ab480facdfefa6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:50.197142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:50.206876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e1e199-e56b-4c50-9b25-5f411708c298 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:50.263482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:50.263827 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.264102 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.264376 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] PATH_INFO: `/v3/users/0f2bddf7c3974b289b5b609eb6d075c0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:50.265485 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2bddf7c3974b289b5b609eb6d075c0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:50.267424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:50.321272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:50.323440 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2bddf7c3974b289b5b609eb6d075c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.332254 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2bddf7c3974b289b5b609eb6d075c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.333048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2bddf7c3974b289b5b609eb6d075c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.334222 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2bddf7c3974b289b5b609eb6d075c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.341290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-25747717-ec65-4063-890a-8d8cc4aefc41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2bddf7c3974b289b5b609eb6d075c0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.359288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1352/2732] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:50 2022] DELETE /identity/v3/users/0f2bddf7c3974b289b5b609eb6d075c0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:50.622155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:50.623804 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:50.636708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6224cb9-0ac2-4345-8a55-a736de77e192 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:50.815211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:50.821632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:50.822481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.822937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.823534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] PATH_INFO: `/v3/projects/5cb681923cee45b4b072c841613df6ae` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:50.824826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cb681923cee45b4b072c841613df6ae)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:50.827808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:50.871995 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cb681923cee45b4b072c841613df6ae', 'request_id': 'req-cdf82129-5328-41b7-9867-8b8ef0c5e102'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.878333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cb681923cee45b4b072c841613df6ae', 'request_id': 'req-cdf82129-5328-41b7-9867-8b8ef0c5e102'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.885355 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cb681923cee45b4b072c841613df6ae', 'request_id': 'req-cdf82129-5328-41b7-9867-8b8ef0c5e102'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.896281 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] The token cache is being invalidate because project 5cb681923cee45b4b072c841613df6ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:50.896565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.897223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-cdf82129-5328-41b7-9867-8b8ef0c5e102 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:50.899067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1381/2733] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:50 2022] DELETE /identity/v3/projects/5cb681923cee45b4b072c841613df6ae => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:50.903735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.904247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.904247 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:50.923904 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:50.925101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:50.982726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1353/2734] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:50.991656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:50.991945 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:50.992215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.011915 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:51.013118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:51.021507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1382/2735] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:51.025547 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.035512 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f91c94-7e2c-491a-be3b-63880c5a9a0b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.040098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.040687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.040962 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.041233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.041911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:51.042214 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.043788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.055153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1354/2736] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:51.059121 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.068625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2551efc-eadc-463a-8cd5-b7a0845d8880 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.073153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.073752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.074042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.074316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.074940 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.077453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.103320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1383/2737] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:51 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:51.107839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.117641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7b55c-22ba-4b67-bfe4-b8751402f67e admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.173062 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.173656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.173971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.174240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.174856 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.176754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.198135 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1355/2738] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:51 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:51.202288 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.212272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a7392-ca5b-476b-a9ce-76bd13459d84 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.216808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.217440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.217714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.218013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.218648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.220227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.229910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1384/2739] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:51.234199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.243967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57944b24-2613-4c9d-8f5a-77c128bdb4ae admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.249319 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c320cbc-a95a-411b-a3ec-2fcb13077de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.249940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c320cbc-a95a-411b-a3ec-2fcb13077de1 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.250233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c320cbc-a95a-411b-a3ec-2fcb13077de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.250498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c320cbc-a95a-411b-a3ec-2fcb13077de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.251130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c320cbc-a95a-411b-a3ec-2fcb13077de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.252696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c320cbc-a95a-411b-a3ec-2fcb13077de1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.263220 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1356/2740] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:51.267275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.276878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d2711d-c81c-4c42-8c78-fb5af9a38880 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.281346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.282008 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.282279 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.282545 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] PATH_INFO: `/v3/projects/abbc9d8f5c494988b9ab090633c2d62a/users/cc1287bca5e2455b922080c78a8668eb/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.297170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=abbc9d8f5c494988b9ab090633c2d62a, user_id=cc1287bca5e2455b922080c78a8668eb, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.299780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.312642 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1385/2741] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:51 2022] PUT /identity/v3/projects/abbc9d8f5c494988b9ab090633c2d62a/users/cc1287bca5e2455b922080c78a8668eb/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:51.322322 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.322600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.322868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.350023 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] MFA Rules not processed for user `cc1287bca5e2455b922080c78a8668eb`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:51.351212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:51.369375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:51.449312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1357/2742] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:51.450548 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.516228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469be199-9208-4329-8cde-6d0b04cb9b7f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.521896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.522492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.522768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.523035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.523658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.525532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.560061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1386/2743] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:51 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:51.563868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.574616 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f21b02-306f-4db1-baec-5f6a2c5ed385 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.633338 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.633982 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.634273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.634523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.635158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.637085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.661488 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1358/2744] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:51 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:51.665826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.676697 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e75d991-542d-4052-841f-1c15ca3d5814 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.681326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.681951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.682229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.682495 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.683126 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.684843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.687125 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:51.696453 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1387/2745] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:51.702565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.712966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841abd01-2137-4bd0-8360-bfb2ba65246d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.719511 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.721013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.721464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.722001 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] PATH_INFO: `/v3/projects/77d450b4184745a5bd7c396b1200a7e3/users/cff005f78e3d408e976ccb4daa635a82/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.746839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d450b4184745a5bd7c396b1200a7e3, user_id=cff005f78e3d408e976ccb4daa635a82, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.751770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.767879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1359/2746] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:51 2022] PUT /identity/v3/projects/77d450b4184745a5bd7c396b1200a7e3/users/cff005f78e3d408e976ccb4daa635a82/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:51.771985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.782693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2effd8-4150-4f31-993a-2f319ca77884 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.847030 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da7f5c-2114-40f5-8466-b7f10f2ed0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.847705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7f5c-2114-40f5-8466-b7f10f2ed0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.848108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7f5c-2114-40f5-8466-b7f10f2ed0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.848334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7f5c-2114-40f5-8466-b7f10f2ed0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.849025 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7f5c-2114-40f5-8466-b7f10f2ed0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.850701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da7f5c-2114-40f5-8466-b7f10f2ed0b6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.860598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1388/2747] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:51.865050 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:51.875735 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f197ed-dfbc-42e8-8cf8-64fcb2f2afec admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:51.880580 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:51.881442 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.881719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.882017 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] PATH_INFO: `/v3/projects/77d450b4184745a5bd7c396b1200a7e3/users/cff005f78e3d408e976ccb4daa635a82/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.883936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77d450b4184745a5bd7c396b1200a7e3, user_id=cff005f78e3d408e976ccb4daa635a82, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:51.886630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:51.899997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1360/2748] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:51 2022] PUT /identity/v3/projects/77d450b4184745a5bd7c396b1200a7e3/users/cff005f78e3d408e976ccb4daa635a82/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:51.912600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:51.912954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:51.913246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:51.943201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] MFA Rules not processed for user `cff005f78e3d408e976ccb4daa635a82`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:51.944459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:52.022531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1389/2749] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:51 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:52.035627 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:52.037526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:52.094776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296d8359-76c1-46eb-9281-297e5b5af0f3 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:52.098292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:52.098878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:52.099162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:52.099431 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:52.108203 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:52.109609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:52.118915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1361/2750] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:52.193267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:52.203418 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd64d10-9860-427f-b547-4dc255164bec None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:52.207085 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5673f-ad18-4a98-b285-d7d353f2033c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:52.208054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5673f-ad18-4a98-b285-d7d353f2033c service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:52.208350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5673f-ad18-4a98-b285-d7d353f2033c service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:52.208625 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5673f-ad18-4a98-b285-d7d353f2033c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:52.265593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5673f-ad18-4a98-b285-d7d353f2033c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:52.267041 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5673f-ad18-4a98-b285-d7d353f2033c service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:52.275941 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1390/2751] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:58.111823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.125517 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:58.176901 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63309cf-c216-479d-ac10-d74086d7815d service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.181463 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.182110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.182378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.182650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] PATH_INFO: `/v3/users/cc1287bca5e2455b922080c78a8668eb` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.183755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1287bca5e2455b922080c78a8668eb)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.186349 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.238635 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1287bca5e2455b922080c78a8668eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.247472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1287bca5e2455b922080c78a8668eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.248048 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1287bca5e2455b922080c78a8668eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.248704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1287bca5e2455b922080c78a8668eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.255518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1287bca5e2455b922080c78a8668eb'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.280983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1362/2752] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:58 2022] DELETE /identity/v3/users/cc1287bca5e2455b922080c78a8668eb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:58.286040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.286508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.286846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.307989 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:58.309194 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.324468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:58.383468 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1391/2753] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:58.394696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.396526 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.458591 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb23d7-c399-44da-9086-4278cacdfefe admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.463356 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.463959 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.464233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.464502 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.474435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.475833 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.486113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1363/2754] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:26:58.775122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.776546 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:58.793537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868ce8ef-44ec-4555-bb02-6d7d67591d2e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.798400 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.799113 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.799313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.799618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] PATH_INFO: `/v3/projects/abbc9d8f5c494988b9ab090633c2d62a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.800699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] RBAC: Authorizing `identity:delete_project(project_id=abbc9d8f5c494988b9ab090633c2d62a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.802707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.850606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abbc9d8f5c494988b9ab090633c2d62a', 'request_id': 'req-c86d2363-6f93-4fec-a5ab-9276a725862d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.856698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abbc9d8f5c494988b9ab090633c2d62a', 'request_id': 'req-c86d2363-6f93-4fec-a5ab-9276a725862d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.863917 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abbc9d8f5c494988b9ab090633c2d62a', 'request_id': 'req-c86d2363-6f93-4fec-a5ab-9276a725862d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.873930 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] The token cache is being invalidate because project abbc9d8f5c494988b9ab090633c2d62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:58.874214 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.874821 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:58.876266 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1392/2755] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:58 2022] DELETE /identity/v3/projects/abbc9d8f5c494988b9ab090633c2d62a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:58.880206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:58.890426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff42753-2045-49b0-86a1-e18d3019a9ea service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:58.923353 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:58.948930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:58.949545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:58.949849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:58.950119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] PATH_INFO: `/v3/users/cff005f78e3d408e976ccb4daa635a82` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:58.951754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cff005f78e3d408e976ccb4daa635a82)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:58.953636 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:58.996479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cff005f78e3d408e976ccb4daa635a82'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.004358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cff005f78e3d408e976ccb4daa635a82'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.005055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cff005f78e3d408e976ccb4daa635a82'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.005652 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cff005f78e3d408e976ccb4daa635a82'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.012372 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ef45a757-aaa8-4a47-bc08-4bb2e64a580e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cff005f78e3d408e976ccb4daa635a82'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.029373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1364/2756] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:26:58 2022] DELETE /identity/v3/users/cff005f78e3d408e976ccb4daa635a82 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:59.294756 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.296212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.309619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d2363-6f93-4fec-a5ab-9276a725862d admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.367115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.367733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.368013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.368278 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] PATH_INFO: `/v3/projects/77d450b4184745a5bd7c396b1200a7e3` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.369376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77d450b4184745a5bd7c396b1200a7e3)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.371282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.414715 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:59.418604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77d450b4184745a5bd7c396b1200a7e3', 'request_id': 'req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.427118 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77d450b4184745a5bd7c396b1200a7e3', 'request_id': 'req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.435927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77d450b4184745a5bd7c396b1200a7e3', 'request_id': 'req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.447310 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] The token cache is being invalidate because project 77d450b4184745a5bd7c396b1200a7e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:26:59.447654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.448226 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f388fcc5-3da4-4d4d-afb6-57b07a7feee7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:26:59.454748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1393/2757] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:26:59 2022] DELETE /identity/v3/projects/77d450b4184745a5bd7c396b1200a7e3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:59.461429 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.461940 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.462030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.481374 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:59.483179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.541434 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1365/2758] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:59.549484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.549800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.550077 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.570026 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:59.571244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.578960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1394/2759] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:59.583133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.592753 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f35134b-1e84-4269-82e3-78b17042926d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.597292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.597923 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.598198 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.598470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.599107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:26:59.599409 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.601011 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.612849 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1366/2760] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:26:59.616903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.626587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd070f-920e-4e6a-a757-1b8c74bb50c3 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.631138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.631713 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.631988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.632256 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.632870 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.634835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.661198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1395/2761] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:26:59.665528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.676113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d09653-8281-4a0f-844c-8cc2f3023584 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.717065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:26:59.735225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.735820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.736094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.736359 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.736981 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.738908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.758465 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1367/2762] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:26:59.762556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.772776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b2021-3d00-495c-a0e1-a973ae73fd83 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.777303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.777939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.778215 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.778485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.779652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.781222 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.791141 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1396/2763] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:26:59.795562 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.805193 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3339b628-93a5-414b-8f03-9a7aaf139c7f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.809654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3e76d1-dd21-40ca-9b82-d86d9f4802b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.810295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e76d1-dd21-40ca-9b82-d86d9f4802b7 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.810567 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e76d1-dd21-40ca-9b82-d86d9f4802b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.810835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3e76d1-dd21-40ca-9b82-d86d9f4802b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.811459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e76d1-dd21-40ca-9b82-d86d9f4802b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.813030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3e76d1-dd21-40ca-9b82-d86d9f4802b7 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.823156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1368/2764] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:26:59.827127 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:26:59.837137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1646754f-a07b-48c6-b6e7-3c9aabbae743 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:26:59.842410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:26:59.843053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.843331 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.843622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] PATH_INFO: `/v3/projects/82f691748dc4469d956e4e649e721a5e/users/eddfc7f7d2dc43b192633bc323ca8659/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.858765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=82f691748dc4469d956e4e649e721a5e, user_id=eddfc7f7d2dc43b192633bc323ca8659, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:26:59.861376 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:26:59.873733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1397/2765] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:26:59 2022] PUT /identity/v3/projects/82f691748dc4469d956e4e649e721a5e/users/eddfc7f7d2dc43b192633bc323ca8659/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:26:59.884097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:26:59.884400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:26:59.884667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:26:59.912330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] MFA Rules not processed for user `eddfc7f7d2dc43b192633bc323ca8659`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:26:59.913514 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:26:59.986219 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1369/2766] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:26:59.991717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.052849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df168e05-854c-4dec-95a6-e6145faa7dfb admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.057341 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.057957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.058234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.058503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.059115 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.061002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.087144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1398/2767] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:26:59 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:00.091136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.102032 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0b5b5-f529-4970-9eac-a432100c220c None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.158518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.159115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.159384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.159648 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.160284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.162188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.182984 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1370/2768] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:00 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:27:00.186898 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.196932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33067e-d037-4c52-a4c1-663f12d8f598 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.201997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.202586 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.202861 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.203133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.203761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.205332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.215590 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1399/2769] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:00.219988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.229762 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748cc1b-29a0-4bd3-978a-5cb1d0df867d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.234290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.234920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.235191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.235474 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] PATH_INFO: `/v3/projects/e28d9bf6d8144518b5a3f6fc75c39faf/users/2ff79216a10b4eab8c4665df66c26bfc/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.252482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28d9bf6d8144518b5a3f6fc75c39faf, user_id=2ff79216a10b4eab8c4665df66c26bfc, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.255405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.268049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1371/2770] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:00 2022] PUT /identity/v3/projects/e28d9bf6d8144518b5a3f6fc75c39faf/users/2ff79216a10b4eab8c4665df66c26bfc/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:00.271977 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.282369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e06f12-694c-4e28-848f-274456f6cec2 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.320969 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:00.338792 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985e25da-66d9-41f4-a850-ed77b0dd93a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.339389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e25da-66d9-41f4-a850-ed77b0dd93a6 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.339666 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e25da-66d9-41f4-a850-ed77b0dd93a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.339936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e25da-66d9-41f4-a850-ed77b0dd93a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.340564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e25da-66d9-41f4-a850-ed77b0dd93a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.342761 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985e25da-66d9-41f4-a850-ed77b0dd93a6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.352191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1400/2771] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:00.356400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.368167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9590ceec-bfde-4053-9c33-dcbbe889d571 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.372977 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.373817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.374133 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.374493 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] PATH_INFO: `/v3/projects/e28d9bf6d8144518b5a3f6fc75c39faf/users/2ff79216a10b4eab8c4665df66c26bfc/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.376594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e28d9bf6d8144518b5a3f6fc75c39faf, user_id=2ff79216a10b4eab8c4665df66c26bfc, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.379200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.389658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1372/2772] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:00 2022] PUT /identity/v3/projects/e28d9bf6d8144518b5a3f6fc75c39faf/users/2ff79216a10b4eab8c4665df66c26bfc/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:00.400770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.401057 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.401336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.430015 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] MFA Rules not processed for user `2ff79216a10b4eab8c4665df66c26bfc`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:00.431213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:00.498168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1401/2773] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:00.509820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.511532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:00.517519 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:00.569345 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d891891-4b1d-47f6-8f5a-e0eee4a8c638 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.573000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a3227-e16c-40eb-8593-0401374abd82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.573524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a3227-e16c-40eb-8593-0401374abd82 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.573820 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a3227-e16c-40eb-8593-0401374abd82 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.574088 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a3227-e16c-40eb-8593-0401374abd82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.583397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a3227-e16c-40eb-8593-0401374abd82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.584790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a3227-e16c-40eb-8593-0401374abd82 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.594083 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1373/2774] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:00.659951 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.722705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857a31f5-9ad4-4532-b4f7-5d3627157f7f None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.727699 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.728332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.728608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.728882 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] PATH_INFO: `/v3/users/eddfc7f7d2dc43b192633bc323ca8659` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.730024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=eddfc7f7d2dc43b192633bc323ca8659)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.731992 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.785227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eddfc7f7d2dc43b192633bc323ca8659'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:00.792333 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eddfc7f7d2dc43b192633bc323ca8659'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:00.792943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eddfc7f7d2dc43b192633bc323ca8659'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:00.793534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eddfc7f7d2dc43b192633bc323ca8659'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:00.800433 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eddfc7f7d2dc43b192633bc323ca8659'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:00.802328 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:00.819132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1402/2775] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:00 2022] DELETE /identity/v3/users/eddfc7f7d2dc43b192633bc323ca8659 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:00.823696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.823917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.824185 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.843678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:00.844909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:00.902232 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1374/2776] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:00.912193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:00.913587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:00.970183 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215fefa-591d-4dea-bb0b-65ea82d4c7fa admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:00.973926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:00.974526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:00.974803 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:00.975075 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:00.984470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:00.985907 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:00.995134 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1403/2777] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:01.252733 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:01.254264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:01.266299 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291c30fd-45cd-46c9-a46a-1f546885fa00 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:01.271175 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:01.271829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:01.272073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:01.272340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] PATH_INFO: `/v3/projects/82f691748dc4469d956e4e649e721a5e` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:01.273604 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=82f691748dc4469d956e4e649e721a5e)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:01.275508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:01.325750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82f691748dc4469d956e4e649e721a5e', 'request_id': 'req-ae974751-7ab4-4baa-b194-7ec65c798f9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.331920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82f691748dc4469d956e4e649e721a5e', 'request_id': 'req-ae974751-7ab4-4baa-b194-7ec65c798f9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.339479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82f691748dc4469d956e4e649e721a5e', 'request_id': 'req-ae974751-7ab4-4baa-b194-7ec65c798f9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.350522 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] The token cache is being invalidate because project 82f691748dc4469d956e4e649e721a5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:01.350801 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.351421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.352902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1375/2778] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:01 2022] DELETE /identity/v3/projects/82f691748dc4469d956e4e649e721a5e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:01.357481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:01.367723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071fb65-11b1-41fd-9874-e8c4c9de4d01 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:01.428322 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:01.428956 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:01.429233 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:01.429505 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] PATH_INFO: `/v3/users/2ff79216a10b4eab8c4665df66c26bfc` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:01.430659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ff79216a10b4eab8c4665df66c26bfc)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:01.432568 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:01.477485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ff79216a10b4eab8c4665df66c26bfc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.485211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ff79216a10b4eab8c4665df66c26bfc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.485923 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ff79216a10b4eab8c4665df66c26bfc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.486516 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ff79216a10b4eab8c4665df66c26bfc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.493191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-96f53c31-56be-4b7e-b107-eb84923ca1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ff79216a10b4eab8c4665df66c26bfc'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.509529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1404/2779] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:01 2022] DELETE /identity/v3/users/2ff79216a10b4eab8c4665df66c26bfc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:01.774621 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:01.777646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:01.792251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae974751-7ab4-4baa-b194-7ec65c798f9c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:01.822153 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:01.859937 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:01.860494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:01.860770 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:01.861041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] PATH_INFO: `/v3/projects/e28d9bf6d8144518b5a3f6fc75c39faf` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:01.862204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e28d9bf6d8144518b5a3f6fc75c39faf)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:01.864064 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:01.909274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e28d9bf6d8144518b5a3f6fc75c39faf', 'request_id': 'req-c5a6f4cd-3d2b-4149-9842-9d864083acf9'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.915343 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e28d9bf6d8144518b5a3f6fc75c39faf', 'request_id': 'req-c5a6f4cd-3d2b-4149-9842-9d864083acf9'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.922521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e28d9bf6d8144518b5a3f6fc75c39faf', 'request_id': 'req-c5a6f4cd-3d2b-4149-9842-9d864083acf9'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.933085 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] The token cache is being invalidate because project e28d9bf6d8144518b5a3f6fc75c39faf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:01.933396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.934096 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c5a6f4cd-3d2b-4149-9842-9d864083acf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:01.936056 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1376/2780] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:01 2022] DELETE /identity/v3/projects/e28d9bf6d8144518b5a3f6fc75c39faf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:01.940835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:01.941123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:01.941395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:01.949769 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:01.962954 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:01.964311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:02.022649 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1405/2781] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:02.030272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.030556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.030828 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.050351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:02.051545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:02.059391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1377/2782] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:02.063926 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.073484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45070bc1-6c1a-499c-9fec-8cca2a7b80b6 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.078654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.079250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.079523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.079789 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.080419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:02.080715 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.082344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.094638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1406/2783] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:27:02.099005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.108404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f94000e-abed-40ba-b9e9-89858c2b8ede None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.115202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.115738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.116040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.116309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.116939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.118855 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.147545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1378/2784] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:02.151711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.161997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d03c93-c97f-4233-9bf9-5cb8b625d8ec admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.216429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.217020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.217297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.217569 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.218225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.220119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.240943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1407/2785] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/users => generated 291 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:02.244949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.255930 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5eb92-7d97-4511-b325-14b5d3422188 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.260500 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.261086 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.261360 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.261630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.262292 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.264481 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.274114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1379/2786] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:02.277944 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.287943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133a9502-5d17-41a4-b9ee-b138156330d9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.292475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874aacdd-c571-413c-9049-1523102ef248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.293056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aacdd-c571-413c-9049-1523102ef248 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.293332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aacdd-c571-413c-9049-1523102ef248 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.293600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874aacdd-c571-413c-9049-1523102ef248 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.294259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aacdd-c571-413c-9049-1523102ef248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.295830 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874aacdd-c571-413c-9049-1523102ef248 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.305991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1408/2787] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:02.310270 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.320005 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeadda55-0801-4576-9886-7c0b96a9b4fd admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.324586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.325217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.326138 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.326405 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] PATH_INFO: `/v3/projects/c4c496675e5444978b457e96da82472a/users/6074b8d885c7421494c5e701926aa714/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.340386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4c496675e5444978b457e96da82472a, user_id=6074b8d885c7421494c5e701926aa714, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.343034 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.355425 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1380/2788] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:02 2022] PUT /identity/v3/projects/c4c496675e5444978b457e96da82472a/users/6074b8d885c7421494c5e701926aa714/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:02.366201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.366659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.366659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.393985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] MFA Rules not processed for user `6074b8d885c7421494c5e701926aa714`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:02.395196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:02.457517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:02.461839 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1409/2789] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:27:02.466306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.522629 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:02.528884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41fa5a-c1cc-48c2-98ba-1172fa689831 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.534806 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.535410 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.535681 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.535976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.536601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.538527 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.565288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1381/2790] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/projects => generated 381 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:02.569308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.580111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32fdbdb-7d5d-439a-a268-a7b18e2ccb72 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.641438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.642004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.642283 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.642551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.643164 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.645052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.665289 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1410/2791] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:02.669721 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.679693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b680ed-872e-41e5-8d86-e7bc687dd0ec admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.684406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.685535 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.685843 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.686113 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.686744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.688303 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.699618 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1382/2792] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:02.703993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.714130 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af405c19-570b-4b5e-959a-6768e268dd10 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.719541 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.720196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.720491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.720819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] PATH_INFO: `/v3/projects/d087154518de45dc9956e5ee83430c0f/users/e5afc4b355864c579aec3079f29a9e9c/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.736034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d087154518de45dc9956e5ee83430c0f, user_id=e5afc4b355864c579aec3079f29a9e9c, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.738696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.750876 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1411/2793] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:02 2022] PUT /identity/v3/projects/d087154518de45dc9956e5ee83430c0f/users/e5afc4b355864c579aec3079f29a9e9c/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:02.754823 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.765068 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f944d-8eee-45e6-836c-f0d541b57736 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.820369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb06a11-73f5-441e-a121-c4f6f0ed255b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.820987 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06a11-73f5-441e-a121-c4f6f0ed255b admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.821216 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06a11-73f5-441e-a121-c4f6f0ed255b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.821482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06a11-73f5-441e-a121-c4f6f0ed255b admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.822147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb06a11-73f5-441e-a121-c4f6f0ed255b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.823707 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb06a11-73f5-441e-a121-c4f6f0ed255b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.834754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1383/2794] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:27:02.839165 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.850315 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231f2d36-4792-4dee-b050-3574af6931d7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:02.854996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:02.855638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.855910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.856177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] PATH_INFO: `/v3/projects/d087154518de45dc9956e5ee83430c0f/users/e5afc4b355864c579aec3079f29a9e9c/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.858758 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d087154518de45dc9956e5ee83430c0f, user_id=e5afc4b355864c579aec3079f29a9e9c, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:02.861332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:02.871744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1412/2795] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:02 2022] PUT /identity/v3/projects/d087154518de45dc9956e5ee83430c0f/users/e5afc4b355864c579aec3079f29a9e9c/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:02.883419 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:02.883703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:02.883974 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:02.911872 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] MFA Rules not processed for user `e5afc4b355864c579aec3079f29a9e9c`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:02.912999 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:02.982484 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1384/2796] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:02.994587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:02.996339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:03.057386 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc152cdb-56a6-4926-a470-6f8983104d8e admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:03.061029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:03.061628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:03.061931 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:03.062199 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:03.071151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:03.072551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:03.082155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1413/2797] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:03.383435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:03.393140 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc7c8d-8815-4255-9e73-7427a79c6a06 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:03.397587 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5afc4b355864c579aec3079f29a9e9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd087154518de45dc9956e5ee83430c0f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:03.398207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:03.398486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:03.398755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:03.399385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:03.399704 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:03.401263 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:03.414911 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1385/2798] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:27:03.418811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:03.427770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171ccb14-0790-487f-872d-f7b3c28c6c5c service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:03.432504 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5afc4b355864c579aec3079f29a9e9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd087154518de45dc9956e5ee83430c0f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:03.433100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:03.433378 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:03.433648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:03.434385 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:03.436788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:03.465685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1414/2799] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:03 2022] POST /identity/v3/projects => generated 354 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:03.470565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:03.480306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb584-211d-492f-aa22-c37265af7170 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:03.513592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:03.538383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5afc4b355864c579aec3079f29a9e9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd087154518de45dc9956e5ee83430c0f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:03.538972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:03.539249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:03.539520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:03.540150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:03.540447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:03.542045 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:03.555098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1386/2800] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 20:27:03.593862 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:03.603904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131bf1d1-5db8-4c12-a4e4-35e04a10efe3 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:03.608455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e5afc4b355864c579aec3079f29a9e9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd087154518de45dc9956e5ee83430c0f', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:03.609071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:03.609354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:03.609626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] PATH_INFO: `/v3/projects/142d3207f8e64193a603d473fdb4a8d5` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:03.610780 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorizing `identity:delete_project(project_id=142d3207f8e64193a603d473fdb4a8d5)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:03.612667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:03.645901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:INSERT=6,keystone:DELETE=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:03.663006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '142d3207f8e64193a603d473fdb4a8d5', 'request_id': 'req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:03.669848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '142d3207f8e64193a603d473fdb4a8d5', 'request_id': 'req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:03.679157 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '142d3207f8e64193a603d473fdb4a8d5', 'request_id': 'req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:03.690819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] The token cache is being invalidate because project 142d3207f8e64193a603d473fdb4a8d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:03.691103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:03.691730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-bc8d98a6-cc28-4e76-a86a-d6be8ec7e307 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:03.693188 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1415/2801] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:03 2022] DELETE /identity/v3/projects/142d3207f8e64193a603d473fdb4a8d5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:03.899179 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:03.901049 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:03.967252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25294b34-7716-4ec0-87cf-019ada8b8043 tempest-VolumeQuotaClassesTest-1551127785 tempest-VolumeQuotaClassesTest-1551127785-project-admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:04.022464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:04.023095 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:04.023373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:04.023646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] PATH_INFO: `/v3/users/6074b8d885c7421494c5e701926aa714` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:04.024777 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6074b8d885c7421494c5e701926aa714)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:04.026730 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:04.083559 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6074b8d885c7421494c5e701926aa714'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.092668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6074b8d885c7421494c5e701926aa714'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.093392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6074b8d885c7421494c5e701926aa714'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.094016 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6074b8d885c7421494c5e701926aa714'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.102000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6074b8d885c7421494c5e701926aa714'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.116070 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:04.123149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1387/2802] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:03 2022] DELETE /identity/v3/users/6074b8d885c7421494c5e701926aa714 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:04.127895 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:04.128198 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:04.128470 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:04.151833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:04.153587 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:04.223011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1416/2803] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:04.229803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:04.232620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:04.296125 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d82d42-6273-4c75-87b6-45478833a2b0 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:04.299594 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:04.300188 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:04.300460 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:04.300725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:04.309693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:04.311115 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:04.320337 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1388/2804] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:04.588778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:04.590241 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:04.603213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5073352-628e-4b3d-a14b-fcc4459dc20d None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:04.608211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:04.608845 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:04.609195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:04.609461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] PATH_INFO: `/v3/projects/c4c496675e5444978b457e96da82472a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:04.610600 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4c496675e5444978b457e96da82472a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:04.612622 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:04.659812 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4c496675e5444978b457e96da82472a', 'request_id': 'req-ece6262d-c16d-4489-8faa-e885971fb0ea'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.665967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4c496675e5444978b457e96da82472a', 'request_id': 'req-ece6262d-c16d-4489-8faa-e885971fb0ea'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.673267 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4c496675e5444978b457e96da82472a', 'request_id': 'req-ece6262d-c16d-4489-8faa-e885971fb0ea'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.683566 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] The token cache is being invalidate because project c4c496675e5444978b457e96da82472a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:04.683855 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.684492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.686095 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1417/2805] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:04 2022] DELETE /identity/v3/projects/c4c496675e5444978b457e96da82472a => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:04.689892 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:04.699468 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dbe675-37da-4875-86b7-42bf5997e1b0 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:04.762132 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:04.763386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:04.763718 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:04.764000 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] PATH_INFO: `/v3/users/e5afc4b355864c579aec3079f29a9e9c` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:04.765189 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5afc4b355864c579aec3079f29a9e9c)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:04.767166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:04.813313 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5afc4b355864c579aec3079f29a9e9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.820568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5afc4b355864c579aec3079f29a9e9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.821237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5afc4b355864c579aec3079f29a9e9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.821969 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5afc4b355864c579aec3079f29a9e9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.829531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5afc4b355864c579aec3079f29a9e9c'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:04.848111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1389/2806] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:04 2022] DELETE /identity/v3/users/e5afc4b355864c579aec3079f29a9e9c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:05.056381 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.057831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.101704 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:05.114775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.116277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.138438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece6262d-c16d-4489-8faa-e885971fb0ea admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.141422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:05.191272 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda07562-dddb-4b35-a7c3-cf7717936641 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.192429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.193034 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.193308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.193574 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.195520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.195905 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.196523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.196799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.197065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] PATH_INFO: `/v3/projects/d087154518de45dc9956e5ee83430c0f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.198204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d087154518de45dc9956e5ee83430c0f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.200094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.247986 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd087154518de45dc9956e5ee83430c0f', 'request_id': 'req-63337e62-eb4c-434a-b48a-0af8aef5e50a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:05.254107 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd087154518de45dc9956e5ee83430c0f', 'request_id': 'req-63337e62-eb4c-434a-b48a-0af8aef5e50a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:05.260037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.261500 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb161103-5b35-4e7a-891e-fc4ae69d8c16 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.263671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd087154518de45dc9956e5ee83430c0f', 'request_id': 'req-63337e62-eb4c-434a-b48a-0af8aef5e50a'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:05.273659 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1418/2807] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:05.277593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] The token cache is being invalidate because project d087154518de45dc9956e5ee83430c0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:05.277932 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:05.278653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-63337e62-eb4c-434a-b48a-0af8aef5e50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:05.280139 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1390/2808] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:05 2022] DELETE /identity/v3/projects/d087154518de45dc9956e5ee83430c0f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:05.289347 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.289781 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.290066 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.309124 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:05.310878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.371076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1419/2809] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:05.379470 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.379750 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.380019 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.400020 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:05.401262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.409761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1391/2810] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:05.414284 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.425003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f551c-f7e3-4713-84df-d576e820202c None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.429705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.430332 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.430610 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.430879 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.431510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:05.431810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.433461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.445841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1420/2811] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:27:05.450336 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.459505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47149ef0-071c-4e55-920f-1ef516676f13 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.464304 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.464912 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.465166 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.465422 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.466183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.468060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.495581 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1392/2812] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:05 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:05.500598 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.512824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdb0385-58e7-47d8-928c-de89682b87f6 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.546914 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:05.581237 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.581893 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.582173 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.582446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.583107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.585067 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.607512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1421/2813] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:05 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:27:05.611284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.621634 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10502a2d-5600-410d-a265-c34a0d43aee2 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.626873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.627533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.627889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.628163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.628799 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.630437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.640182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1393/2814] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:05.643980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.655092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a14e1-6bb2-4beb-baa3-3e4e8e40387b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.662399 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b835e-e47f-4728-8026-0b4454fa7696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.662997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b835e-e47f-4728-8026-0b4454fa7696 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.663272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b835e-e47f-4728-8026-0b4454fa7696 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.663573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b835e-e47f-4728-8026-0b4454fa7696 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.664407 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b835e-e47f-4728-8026-0b4454fa7696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.666137 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b835e-e47f-4728-8026-0b4454fa7696 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.678213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1422/2815] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:05.682332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.692754 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7474ad8c-6b7a-4aab-8928-d5df1260ab41 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.697249 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.697915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.698191 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.698456 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] PATH_INFO: `/v3/projects/1d0bd9bb21544e1599302da13026dfb6/users/0313d1a3631946888f79d779a03e39ee/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.712964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0bd9bb21544e1599302da13026dfb6, user_id=0313d1a3631946888f79d779a03e39ee, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.715592 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.728626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1394/2816] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:05 2022] PUT /identity/v3/projects/1d0bd9bb21544e1599302da13026dfb6/users/0313d1a3631946888f79d779a03e39ee/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:05.739577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.739901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.740212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.767254 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] MFA Rules not processed for user `0313d1a3631946888f79d779a03e39ee`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:05.768370 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.835510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1423/2817] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:05 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:05.849028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:05.850863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:05.904650 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:05.911248 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dbdff6-046f-451a-b8c4-16aa9df84cbd admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:05.915868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4af361e-d1ea-4351-bab8-4e51d1b5fee7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:05.916466 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af361e-d1ea-4351-bab8-4e51d1b5fee7 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:05.916744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af361e-d1ea-4351-bab8-4e51d1b5fee7 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:05.917015 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4af361e-d1ea-4351-bab8-4e51d1b5fee7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:05.926009 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af361e-d1ea-4351-bab8-4e51d1b5fee7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:05.927423 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4af361e-d1ea-4351-bab8-4e51d1b5fee7 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:05.936110 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1395/2818] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:05 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:15.827790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:15.932416 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:17.595735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:17.597365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:17.664287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004ac44-9457-482a-81fa-49215e1041ff None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:17.668819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:17.669445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:17.669723 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:17.670022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] PATH_INFO: `/v3/users/5cfbfc0235ca44f1a717d35f65d1e7f4` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:17.671196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cfbfc0235ca44f1a717d35f65d1e7f4)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:17.673108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:17.722459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cfbfc0235ca44f1a717d35f65d1e7f4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:17.730090 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cfbfc0235ca44f1a717d35f65d1e7f4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:17.730717 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cfbfc0235ca44f1a717d35f65d1e7f4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:17.731395 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cfbfc0235ca44f1a717d35f65d1e7f4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:17.738868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cfbfc0235ca44f1a717d35f65d1e7f4'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:17.757146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1424/2819] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:17 2022] DELETE /identity/v3/users/5cfbfc0235ca44f1a717d35f65d1e7f4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:17.761614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:17.762121 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:17.762210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:17.787780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:17.789038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:17.850523 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1396/2820] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:17.860897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:17.862345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:17.926835 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0dce71-7b00-466e-9faa-aa01ec72736c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:17.930475 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:17.931078 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:17.931360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:17.931628 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:17.941582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:17.943039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:17.953553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1425/2821] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:18.208124 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.209528 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.223935 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3d4d7-6a32-4c69-9750-717274502254 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:18.229223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:18.229903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.230297 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.230429 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.230579 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] PATH_INFO: `/v3/users/0313d1a3631946888f79d779a03e39ee` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.231790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0313d1a3631946888f79d779a03e39ee)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:18.232207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.234221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:18.251420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e90ad-fa4d-42af-87db-0710ad667b5e service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:18.256938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:18.257576 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.257936 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.258313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] PATH_INFO: `/v3/projects/c0ad02b2e16e4ca98e9d773005a177df` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.259681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ad02b2e16e4ca98e9d773005a177df)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:18.261901 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:18.279708 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:18.297384 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0313d1a3631946888f79d779a03e39ee'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.305202 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0313d1a3631946888f79d779a03e39ee'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.305887 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0313d1a3631946888f79d779a03e39ee'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.306734 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0313d1a3631946888f79d779a03e39ee'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.314749 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-9b813778-9002-461a-8bd2-c46631cdc6f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0313d1a3631946888f79d779a03e39ee'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.316244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ad02b2e16e4ca98e9d773005a177df', 'request_id': 'req-4c512946-babc-437a-ab01-3c5f9cd8a69a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.322735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ad02b2e16e4ca98e9d773005a177df', 'request_id': 'req-4c512946-babc-437a-ab01-3c5f9cd8a69a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.331570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ad02b2e16e4ca98e9d773005a177df', 'request_id': 'req-4c512946-babc-437a-ab01-3c5f9cd8a69a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.333315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1397/2822] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:18 2022] DELETE /identity/v3/users/0313d1a3631946888f79d779a03e39ee => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:18.337593 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.337978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.338327 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.343259 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] The token cache is being invalidate because project c0ad02b2e16e4ca98e9d773005a177df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:18.343550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.344185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.345638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1426/2823] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:18 2022] DELETE /identity/v3/projects/c0ad02b2e16e4ca98e9d773005a177df => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:18.353359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.354733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.359027 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:18.360217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.425089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1398/2824] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:27:18.428313 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c512946-babc-437a-ab01-3c5f9cd8a69a admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:18.433187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:18.433856 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.434142 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.434420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] PATH_INFO: `/v3/users/c60e2c8b6f8041f897dca94e49c30243` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.434590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.435564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60e2c8b6f8041f897dca94e49c30243)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:18.435956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.438037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:18.452393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:18.491170 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60e2c8b6f8041f897dca94e49c30243'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.500148 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60e2c8b6f8041f897dca94e49c30243'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.500765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60e2c8b6f8041f897dca94e49c30243'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.501410 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60e2c8b6f8041f897dca94e49c30243'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.502093 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ae954d-d01a-42ea-a942-534dd5f7ed86 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:18.505868 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:18.506536 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.506876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.507233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.508963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.509071 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60e2c8b6f8041f897dca94e49c30243'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.521956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:18.523447 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efbb244-718f-4dfe-87e9-ce4337743d42 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:18.531024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1427/2825] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:18 2022] DELETE /identity/v3/users/c60e2c8b6f8041f897dca94e49c30243 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:18.534256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1399/2826] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:27:18.664492 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.666133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.696415 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:18.737810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d2e28-a869-40f5-ad5c-0b922a75fb27 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:18.742588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:18.743223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.743501 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.743772 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] PATH_INFO: `/v3/users/7b6f69d99c2342b4a93aa124d20d499d` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.745080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6f69d99c2342b4a93aa124d20d499d)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:18.747019 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:18.823242 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6f69d99c2342b4a93aa124d20d499d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.836451 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6f69d99c2342b4a93aa124d20d499d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.837314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6f69d99c2342b4a93aa124d20d499d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.838076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6f69d99c2342b4a93aa124d20d499d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.852552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6f69d99c2342b4a93aa124d20d499d'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:18.877042 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1428/2827] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:18 2022] DELETE /identity/v3/users/7b6f69d99c2342b4a93aa124d20d499d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:18.881404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:18.881924 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:18.882002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:18.913427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:18.915273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:18.965776 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:18.968449 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.011251 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1400/2828] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:27:19.015014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.015014 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.031637 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047273e-53bc-4f20-8650-c81708d6473b None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.036702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.037397 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.037755 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.038108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] PATH_INFO: `/v3/projects/1d0bd9bb21544e1599302da13026dfb6` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.039334 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0bd9bb21544e1599302da13026dfb6)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.041330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.058884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a833d79c-4bc2-4eb3-814d-526dd20e9a1c admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.063638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.064261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.064536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.064808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] PATH_INFO: `/v3/projects/fba7a415749a4e3f941341201fbe85a1` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.065948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba7a415749a4e3f941341201fbe85a1)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.067848 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.091699 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0bd9bb21544e1599302da13026dfb6', 'request_id': 'req-8911e1c1-8e0e-46f2-9761-19ebd7262d85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.102845 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0bd9bb21544e1599302da13026dfb6', 'request_id': 'req-8911e1c1-8e0e-46f2-9761-19ebd7262d85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.111053 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0bd9bb21544e1599302da13026dfb6', 'request_id': 'req-8911e1c1-8e0e-46f2-9761-19ebd7262d85'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.128738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:19.128993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] The token cache is being invalidate because project 1d0bd9bb21544e1599302da13026dfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:19.129186 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.129971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.131847 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba7a415749a4e3f941341201fbe85a1', 'request_id': 'req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.132256 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1401/2829] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:19 2022] DELETE /identity/v3/projects/1d0bd9bb21544e1599302da13026dfb6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:19.133098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.134435 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.150007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba7a415749a4e3f941341201fbe85a1', 'request_id': 'req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.173213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba7a415749a4e3f941341201fbe85a1', 'request_id': 'req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.187814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] The token cache is being invalidate because project fba7a415749a4e3f941341201fbe85a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:19.188024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.188640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-fb3a8791-da9e-4bf0-ab1e-1d527b53aef8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.190444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1429/2830] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:18 2022] DELETE /identity/v3/projects/fba7a415749a4e3f941341201fbe85a1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:19.196151 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a000f-63ce-4bd4-a91e-c3bd0042cfe7 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.196469 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a000f-63ce-4bd4-a91e-c3bd0042cfe7 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.196696 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a000f-63ce-4bd4-a91e-c3bd0042cfe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.198826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:19.221781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911e1c1-8e0e-46f2-9761-19ebd7262d85 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.222154 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-f81a000f-63ce-4bd4-a91e-c3bd0042cfe7 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:19.223294 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f81a000f-63ce-4bd4-a91e-c3bd0042cfe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.284403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.285358 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.285907 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.286257 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.288261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.301881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1430/2831] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:27:19.304143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.304436 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.304707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.310506 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.312478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.323264 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1402/2832] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:19.330592 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:19.331929 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.343517 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1431/2833] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:19.344702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.357788 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372445cb-f6b2-4c60-9cb6-92cd0ec25837 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.362285 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.362949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.363225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.363494 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] PATH_INFO: `/v3/domains` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.364158 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93113) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:19.364464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.366162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.380291 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1403/2834] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:27:19.385557 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.396633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6751e-1ce0-49e0-a052-97c624f09420 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.404533 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.406074 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.406290 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.406698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.407438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.410244 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.443943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1432/2835] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:19.448725 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.460676 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05241ce1-01a9-47e1-af47-2b7f6ab06a65 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.522989 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.523693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.524836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.525129 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.526012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.527976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.555368 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1404/2836] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:19.556836 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.569997 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3f37f0-c384-425c-add0-2e55c486ff37 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.575104 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.575866 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.576223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.576643 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.577416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.579363 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.589718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1433/2837] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:19.594277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.605163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649607b6-ec90-47c2-83bd-960b6b8f7679 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.610997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.611679 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.611994 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.612253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.612920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.614537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.626693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1405/2838] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:19.633463 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.633664 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.633777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.643437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18089efe-29d4-4ff7-929f-622652828d4d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.646711 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eefc3d2-bcec-4e6e-b0f2-9b25a3e2bf13 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.648213 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1947853-8d3b-436c-95a4-44b01e73a2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.648961 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947853-8d3b-436c-95a4-44b01e73a2d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.649146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947853-8d3b-436c-95a4-44b01e73a2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.649452 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1947853-8d3b-436c-95a4-44b01e73a2d0 admin admin] PATH_INFO: `/v3/projects/00e46fc3c75344e6a2f21565f08e4b95/users/2c10219ee66d4ab5ab256bfddafb3b77/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.652089 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.652719 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.652994 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.653261 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] PATH_INFO: `/v3/projects/c9d02ad902bd4e50a7f65239e84e33c7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.654452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9d02ad902bd4e50a7f65239e84e33c7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.656445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.671891 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947853-8d3b-436c-95a4-44b01e73a2d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00e46fc3c75344e6a2f21565f08e4b95, user_id=2c10219ee66d4ab5ab256bfddafb3b77, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.675252 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1947853-8d3b-436c-95a4-44b01e73a2d0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.689098 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1406/2839] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:19 2022] PUT /identity/v3/projects/00e46fc3c75344e6a2f21565f08e4b95/users/2c10219ee66d4ab5ab256bfddafb3b77/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:19.702746 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.703239 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.703534 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.718021 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9d02ad902bd4e50a7f65239e84e33c7', 'request_id': 'req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.724990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9d02ad902bd4e50a7f65239e84e33c7', 'request_id': 'req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.733750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9d02ad902bd4e50a7f65239e84e33c7', 'request_id': 'req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.736991 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] MFA Rules not processed for user `2c10219ee66d4ab5ab256bfddafb3b77`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:19.738483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.749550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] The token cache is being invalidate because project c9d02ad902bd4e50a7f65239e84e33c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:19.749877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.750535 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:19.752006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1434/2840] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:19 2022] DELETE /identity/v3/projects/c9d02ad902bd4e50a7f65239e84e33c7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:19.771295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:19.821144 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1407/2841] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:27:19.822459 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.823902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:19.847343 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:19.899514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e7ff0-7c8b-465b-8753-4f7450a2d938 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:19.904652 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:19.906144 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:19.906425 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:19.906700 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:19.907345 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:19.909582 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:19.944487 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1435/2842] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:27:19.945633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:19.957632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d86cc5-dede-4d4d-a5e1-58f57de5f9c2 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.022651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.023261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.023537 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.023808 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.024457 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.026393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.052286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1408/2843] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:19 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:27:20.056207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.067054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594f18b2-1cb3-4109-87b7-71c88781523f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.071841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.072427 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.072698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.072961 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.073585 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.075187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.087270 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1436/2844] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:27:20.091283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.101657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b581455-85fa-4cc7-8cf5-307fc0636f74 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.106274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b61095-6e8d-44e1-b278-ea15d195cc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.106870 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b61095-6e8d-44e1-b278-ea15d195cc6b admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.107145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b61095-6e8d-44e1-b278-ea15d195cc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.107418 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b61095-6e8d-44e1-b278-ea15d195cc6b admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.108084 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b61095-6e8d-44e1-b278-ea15d195cc6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.109669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b61095-6e8d-44e1-b278-ea15d195cc6b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.119611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1409/2845] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:20.124624 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.134949 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebec6638-a0ed-4d95-a8e5-862b7b18e991 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.140128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.140767 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.141038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.141306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] PATH_INFO: `/v3/projects/06112a4a92984183931ecf7e752b31d0/users/ed04926bd1dd43d790a3715cee540464/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.155650 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06112a4a92984183931ecf7e752b31d0, user_id=ed04926bd1dd43d790a3715cee540464, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.158579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.172205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1437/2846] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:20 2022] PUT /identity/v3/projects/06112a4a92984183931ecf7e752b31d0/users/ed04926bd1dd43d790a3715cee540464/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:20.188723 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.189012 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.189286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.217503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] MFA Rules not processed for user `ed04926bd1dd43d790a3715cee540464`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:20.218779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:20.285507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1410/2847] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:20.290181 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.352286 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff2410-b869-415a-b480-62ffda97ec35 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.356886 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.357488 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.357787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.358086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.358718 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.360596 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.387080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1438/2848] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:20 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:20.391976 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.402771 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9788329d-5232-4bed-a80a-35f2d6a3096f None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.435938 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:20.460459 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.461059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.461335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.461608 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] PATH_INFO: `/v3/users` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.462265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.464154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.485826 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1411/2849] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:20 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:20.490040 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.500316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3b881-25d3-4ba6-ab26-3ed5bd6d95f0 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.505167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.505782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.506092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.506365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.507002 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.508618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.520778 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1439/2850] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:20.524550 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.535532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dee1ed-546a-4f32-b097-3ceaecc1d954 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.540196 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.540863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.541141 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.541414 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] PATH_INFO: `/v3/projects/5f9f8a60f4644f869ed19d83eb3e4219/users/9bd76d2e773d4a769ae830c18feb7f4a/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.556167 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9f8a60f4644f869ed19d83eb3e4219, user_id=9bd76d2e773d4a769ae830c18feb7f4a, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.558853 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.572217 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1412/2851] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:20 2022] PUT /identity/v3/projects/5f9f8a60f4644f869ed19d83eb3e4219/users/9bd76d2e773d4a769ae830c18feb7f4a/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:20.576128 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.587049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962b33f-8915-4863-990d-945eaa3cd63b admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.632004 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:20.645687 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d1cd4-4753-4fe2-9813-c700908c67fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.646320 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1cd4-4753-4fe2-9813-c700908c67fb admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.646599 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1cd4-4753-4fe2-9813-c700908c67fb admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.646869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d1cd4-4753-4fe2-9813-c700908c67fb admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.647507 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1cd4-4753-4fe2-9813-c700908c67fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.649083 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d1cd4-4753-4fe2-9813-c700908c67fb admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.659555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1440/2852] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:20 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:20.663741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.675660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a599cf-f41d-4aa7-835e-177bb11edc14 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.680312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.680966 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.681238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.681504 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] PATH_INFO: `/v3/projects/5f9f8a60f4644f869ed19d83eb3e4219/users/9bd76d2e773d4a769ae830c18feb7f4a/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.683532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f9f8a60f4644f869ed19d83eb3e4219, user_id=9bd76d2e773d4a769ae830c18feb7f4a, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.686715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.697155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1413/2853] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:20 2022] PUT /identity/v3/projects/5f9f8a60f4644f869ed19d83eb3e4219/users/9bd76d2e773d4a769ae830c18feb7f4a/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:20.708123 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.708408 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.708683 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.739086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] MFA Rules not processed for user `9bd76d2e773d4a769ae830c18feb7f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:20.740344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:20.809397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1441/2854] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:20 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:20.821656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:20.823391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:20.881993 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d2d4c-bedd-410f-8720-a1f80a11f8ef admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:20.885669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:20.886240 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:20.886520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:20.886804 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:20.944117 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:20.945551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:20.955118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1414/2855] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:22.473553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:22.483466 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cc8250-cb2f-44b7-8382-4369c871b998 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:22.487048 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:22.487660 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:22.487934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:22.488201 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:22.544564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:22.546508 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:22.556390 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1442/2856] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:23.107751 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:23.117237 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801d3fe4-f911-4315-9ec7-dd4783ed6925 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:23.120972 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400387aa-7740-48ca-8409-454c4328f162 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:23.121568 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400387aa-7740-48ca-8409-454c4328f162 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:23.121880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400387aa-7740-48ca-8409-454c4328f162 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:23.122148 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400387aa-7740-48ca-8409-454c4328f162 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:23.131936 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400387aa-7740-48ca-8409-454c4328f162 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:23.133309 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400387aa-7740-48ca-8409-454c4328f162 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:23.143242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1415/2857] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:28.062359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:28.125694 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72326a2-7fd9-4e50-aee7-981286563da4 service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:28.130943 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:28.131572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:28.131875 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:28.132701 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] PATH_INFO: `/v3/users/2c10219ee66d4ab5ab256bfddafb3b77` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:28.133872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c10219ee66d4ab5ab256bfddafb3b77)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:28.135846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:28.138314 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:28.183567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c10219ee66d4ab5ab256bfddafb3b77'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.190670 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c10219ee66d4ab5ab256bfddafb3b77'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.191268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c10219ee66d4ab5ab256bfddafb3b77'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.191863 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c10219ee66d4ab5ab256bfddafb3b77'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.198788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c10219ee66d4ab5ab256bfddafb3b77'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.215356 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1443/2858] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:28 2022] DELETE /identity/v3/users/2c10219ee66d4ab5ab256bfddafb3b77 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:28.220226 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:28.220669 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:28.220781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:28.239946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:28.241154 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:28.265882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:28.299950 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1416/2859] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:28.309703 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:28.311145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:28.365846 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf4030-ff30-4779-ab41-b34a997d71bb admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:28.369382 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:28.369991 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:28.370263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:28.370526 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:28.379327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:28.380672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:28.390455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1444/2860] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:28.661830 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:28.663274 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:28.675963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ebacb4-c9c5-46e2-9e2b-ddaadc9c2b01 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:28.680941 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:28.681582 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:28.681895 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:28.682170 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] PATH_INFO: `/v3/projects/00e46fc3c75344e6a2f21565f08e4b95` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:28.683296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00e46fc3c75344e6a2f21565f08e4b95)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:28.685199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:28.733180 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00e46fc3c75344e6a2f21565f08e4b95', 'request_id': 'req-bd319998-5486-44a3-ae4d-81a22ff8a573'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.739273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00e46fc3c75344e6a2f21565f08e4b95', 'request_id': 'req-bd319998-5486-44a3-ae4d-81a22ff8a573'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.746306 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00e46fc3c75344e6a2f21565f08e4b95', 'request_id': 'req-bd319998-5486-44a3-ae4d-81a22ff8a573'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.756498 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] The token cache is being invalidate because project 00e46fc3c75344e6a2f21565f08e4b95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:28.756781 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.757398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.758882 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1417/2861] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:28 2022] DELETE /identity/v3/projects/00e46fc3c75344e6a2f21565f08e4b95 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:28.762860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:28.772909 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb0003-8a19-493c-884d-109cdcd8b700 service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:28.830032 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:28.830658 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:28.830934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:28.831228 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] PATH_INFO: `/v3/users/ed04926bd1dd43d790a3715cee540464` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:28.832330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed04926bd1dd43d790a3715cee540464)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:28.834262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:28.862817 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:28.882413 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed04926bd1dd43d790a3715cee540464'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.889784 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed04926bd1dd43d790a3715cee540464'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.890389 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed04926bd1dd43d790a3715cee540464'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.890985 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed04926bd1dd43d790a3715cee540464'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.897604 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed04926bd1dd43d790a3715cee540464'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:28.914972 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1445/2862] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:28 2022] DELETE /identity/v3/users/ed04926bd1dd43d790a3715cee540464 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:29.181004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:29.182433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:29.196355 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd319998-5486-44a3-ae4d-81a22ff8a573 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:29.253861 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:29.254483 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:29.254738 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:29.255006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] PATH_INFO: `/v3/projects/06112a4a92984183931ecf7e752b31d0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:29.256094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06112a4a92984183931ecf7e752b31d0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:29.258104 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:29.304151 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06112a4a92984183931ecf7e752b31d0', 'request_id': 'req-84d1e2e0-4300-4332-9282-df1e2a92cfa0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.310122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06112a4a92984183931ecf7e752b31d0', 'request_id': 'req-84d1e2e0-4300-4332-9282-df1e2a92cfa0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.313199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:29.319033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06112a4a92984183931ecf7e752b31d0', 'request_id': 'req-84d1e2e0-4300-4332-9282-df1e2a92cfa0'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.330760 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] The token cache is being invalidate because project 06112a4a92984183931ecf7e752b31d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:29.331150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.331773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.333242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1418/2863] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:29 2022] DELETE /identity/v3/projects/06112a4a92984183931ecf7e752b31d0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:29.337131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:29.338552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:29.351131 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daf38a8-7873-4df3-8ffa-00a06f4749f2 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:29.406521 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:29.407138 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:29.407411 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:29.407677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] PATH_INFO: `/v3/users/9bd76d2e773d4a769ae830c18feb7f4a` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:29.409016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd76d2e773d4a769ae830c18feb7f4a)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:29.410968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:29.460219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd76d2e773d4a769ae830c18feb7f4a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.467675 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd76d2e773d4a769ae830c18feb7f4a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.468335 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd76d2e773d4a769ae830c18feb7f4a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.469038 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd76d2e773d4a769ae830c18feb7f4a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.476330 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-c3427d07-5585-40ee-9fd1-e244f4d2e39d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd76d2e773d4a769ae830c18feb7f4a'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.494206 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1446/2864] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:29 2022] DELETE /identity/v3/users/9bd76d2e773d4a769ae830c18feb7f4a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:29.762480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:29.763953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:29.777764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1e2e0-4300-4332-9282-df1e2a92cfa0 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:29.843486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:29.844119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:29.844398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:29.844667 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] PATH_INFO: `/v3/projects/5f9f8a60f4644f869ed19d83eb3e4219` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:29.845811 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f9f8a60f4644f869ed19d83eb3e4219)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:29.847776 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:29.894561 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f9f8a60f4644f869ed19d83eb3e4219', 'request_id': 'req-21a4270d-75fc-4fdf-a292-59004fa81c3b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.900737 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f9f8a60f4644f869ed19d83eb3e4219', 'request_id': 'req-21a4270d-75fc-4fdf-a292-59004fa81c3b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.908668 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f9f8a60f4644f869ed19d83eb3e4219', 'request_id': 'req-21a4270d-75fc-4fdf-a292-59004fa81c3b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.918878 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] The token cache is being invalidate because project 5f9f8a60f4644f869ed19d83eb3e4219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:29.919163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.919779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-21a4270d-75fc-4fdf-a292-59004fa81c3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:29.921803 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1419/2865] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:29 2022] DELETE /identity/v3/projects/5f9f8a60f4644f869ed19d83eb3e4219 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:29.926900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:29.927212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:29.927483 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:29.946802 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:29.948020 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:29.964481 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:30.007285 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1447/2866] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:30.015510 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.015825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.016089 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.036206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:30.037399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:30.045464 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1420/2867] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:30.049512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.059082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf67e35-92df-4be9-b997-5deae2f4f2a4 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.063679 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.064272 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.064550 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.064820 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.065454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:30.065773 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.067350 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.079013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1448/2868] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:27:30.083030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.092728 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a136bbf-89a5-4659-bd17-8d2ff741ce54 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.097390 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.098004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.098284 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.098556 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.099183 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.101047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.127697 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1421/2869] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:30 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:30.131536 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.141834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d905fe72-5321-4f50-a58e-5901863327c9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.196503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.197105 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.197384 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.197656 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.198311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.200218 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.222520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1449/2870] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:30 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:30.226586 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.236711 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70d4cf-7e58-4ce7-a95b-9242a83175c9 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.241288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.241915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.242197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.242467 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.243109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.244731 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.254437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1422/2871] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:30.258209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.268444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76003b-3746-4ec5-a0f1-2c5ca6354801 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.273037 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469bc7c6-bff6-4a3f-a3a6-c5cb46abe629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.273620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc7c6-bff6-4a3f-a3a6-c5cb46abe629 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.273932 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc7c6-bff6-4a3f-a3a6-c5cb46abe629 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.274207 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469bc7c6-bff6-4a3f-a3a6-c5cb46abe629 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.274864 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bc7c6-bff6-4a3f-a3a6-c5cb46abe629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.276456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469bc7c6-bff6-4a3f-a3a6-c5cb46abe629 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.287635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1450/2872] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:30.291441 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.301503 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b14099-4303-43ab-87fc-8ff72470414c admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.306055 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.306687 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.306970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.307245 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] PATH_INFO: `/v3/projects/cd434e2ee3bd466abe5df77da23961f8/users/03d9fa985d7343528e60455e0d00da65/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.321605 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd434e2ee3bd466abe5df77da23961f8, user_id=03d9fa985d7343528e60455e0d00da65, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.324269 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.337290 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1423/2873] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:30 2022] PUT /identity/v3/projects/cd434e2ee3bd466abe5df77da23961f8/users/03d9fa985d7343528e60455e0d00da65/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:30.343063 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c13fd-7fe8-41dd-be3e-17e3d29e619d None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.343540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c13fd-7fe8-41dd-be3e-17e3d29e619d None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.343633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c13fd-7fe8-41dd-be3e-17e3d29e619d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.362029 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-4c5c13fd-7fe8-41dd-be3e-17e3d29e619d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:30.363229 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4c5c13fd-7fe8-41dd-be3e-17e3d29e619d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:30.420662 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1451/2874] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:30.430008 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:30.431678 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:30.445261 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:30.490255 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ccaad-543e-4f9a-92f9-1bb42ead87de admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:30.493876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:30.494469 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:30.494744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:30.495013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:30.504548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:30.506101 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:30.516169 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1424/2875] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:34.119103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.120263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.120542 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.149174 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] MFA Rules not processed for user `03d9fa985d7343528e60455e0d00da65`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:34.150419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:34.168633 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:34.227460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1452/2876] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:34.231705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:34.244450 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232f410f-867a-40f0-8697-30c26f5d979d service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:34.251430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:34.252042 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.252317 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.252588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.253242 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:34.255197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:34.286282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1425/2877] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:34 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:34.290448 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:34.301603 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c618832-1323-44f9-badd-b9fd218a5016 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:34.398910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:34.399573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.399834 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.400103 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.400745 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:34.402752 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:34.425790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1453/2878] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:34 2022] POST /identity/v3/users => generated 298 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:27:34.429253 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:34.441610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45675a3e-38d1-4969-a88f-2abb1cfb6a1d admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:34.446501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:34.447114 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.447385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.447653 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.448300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:34.449926 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:34.462004 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1426/2879] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 20:27:34.466062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:34.476532 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91f33e1-3d6e-4e47-b614-61ab06d93df3 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:34.482860 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:34.483981 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.484480 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.484980 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] PATH_INFO: `/v3/projects/d6bc4c61c4b549cf99d6bae637ba5641/users/6d86d3d135bd48ed83064e675a48f5e8/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.509282 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc4c61c4b549cf99d6bae637ba5641, user_id=6d86d3d135bd48ed83064e675a48f5e8, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:34.512265 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:34.531438 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1454/2880] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:34 2022] PUT /identity/v3/projects/d6bc4c61c4b549cf99d6bae637ba5641/users/6d86d3d135bd48ed83064e675a48f5e8/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:34.535656 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:34.546262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce79af-06ac-4e5c-b09d-55eff786b988 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:34.604473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff61aa20-ef1f-443c-9405-40c85fef280d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:34.605177 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61aa20-ef1f-443c-9405-40c85fef280d admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.605520 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61aa20-ef1f-443c-9405-40c85fef280d admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.605908 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff61aa20-ef1f-443c-9405-40c85fef280d admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.606620 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61aa20-ef1f-443c-9405-40c85fef280d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:34.608774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff61aa20-ef1f-443c-9405-40c85fef280d admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:34.619532 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1427/2881] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:34.624393 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:34.639046 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6770a-c1e8-494a-9bf4-5a507610bc7f admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:34.643816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:34.644477 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:34.644753 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:34.645022 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] PATH_INFO: `/v3/projects/d6bc4c61c4b549cf99d6bae637ba5641/users/6d86d3d135bd48ed83064e675a48f5e8/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:34.647006 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6bc4c61c4b549cf99d6bae637ba5641, user_id=6d86d3d135bd48ed83064e675a48f5e8, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:34.649795 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:34.661262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1455/2882] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:34 2022] PUT /identity/v3/projects/d6bc4c61c4b549cf99d6bae637ba5641/users/6d86d3d135bd48ed83064e675a48f5e8/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:38.823437 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714162-cd4d-48a9-a9e6-c88424c15eef None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:38.823916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714162-cd4d-48a9-a9e6-c88424c15eef None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:38.823995 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d714162-cd4d-48a9-a9e6-c88424c15eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:38.855854 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-3d714162-cd4d-48a9-a9e6-c88424c15eef None None] MFA Rules not processed for user `6d86d3d135bd48ed83064e675a48f5e8`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:38.857069 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-3d714162-cd4d-48a9-a9e6-c88424c15eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:38.936916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1428/2883] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:38 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:38.952903 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:38.955787 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:39.020490 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75f351-a758-4400-bba0-b4920f0703b8 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:39.025611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c21dea41dba24bf88466a7c3f00b8c2b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:39.026366 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:39.026706 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:39.027068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:39.056024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:39.100476 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:39.102155 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:39.127573 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1456/2884] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:39.304838 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b522d9c-affc-4ee3-adb7-38b5b77842fd None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:39.305182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b522d9c-affc-4ee3-adb7-38b5b77842fd None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:39.305521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b522d9c-affc-4ee3-adb7-38b5b77842fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:39.311789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:39.371172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-0b522d9c-affc-4ee3-adb7-38b5b77842fd None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:39.372394 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-0b522d9c-affc-4ee3-adb7-38b5b77842fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:39.442654 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1429/2885] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 20:27:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:39.491842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:39.493274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:39.586538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e024d1b-b5b6-426b-9f60-67f0f697e68e service glance] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:39.592052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:39.592648 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:39.592924 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:39.593196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:39.600937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:39.602534 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:39.610934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1457/2886] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:27:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:27:48.195378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:48.195902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:48.195902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:48.219701 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] MFA Rules not processed for user `03d9fa985d7343528e60455e0d00da65`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:48.220916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:48.227881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1430/2887] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:48.742678 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:48.801644 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc9c421-f2b5-44b4-b3fd-8831992c4e29 service swift] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:48.805431 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:48.805868 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:48.806287 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:48.806388 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:48.808263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:48.818101 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:48.819619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cf7080-6a89-4075-963b-b16c88a30007 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:48.828452 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1458/2888] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:49.142369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:49.143719 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:49.214178 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7263e2-8ef0-493c-a7f4-c709d2e5cd7d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:49.219094 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:49.219685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:49.219963 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:49.220234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:49.221978 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:49.233052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:49.234660 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:49.244406 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1431/2889] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:50.643698 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:50.644223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:50.644223 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:50.665177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:50.666446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:50.674446 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1459/2890] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 20:27:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:50.687171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:50.701386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac437f7-a125-4f94-87ac-06f3378a0851 service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:50.707207 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa18d051-11e7-48c4-a8ee-d5598ab14264 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:50.707884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d051-11e7-48c4-a8ee-d5598ab14264 service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:50.708223 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d051-11e7-48c4-a8ee-d5598ab14264 service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:50.708688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa18d051-11e7-48c4-a8ee-d5598ab14264 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:50.719386 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18d051-11e7-48c4-a8ee-d5598ab14264 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:50.721375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa18d051-11e7-48c4-a8ee-d5598ab14264 service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:50.732119 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1432/2891] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:27:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:27:51.918588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:51.920145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:51.959578 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=100 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:51.992145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc70f5b-a5c3-400e-8963-b531f94b2bc2 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:51.998112 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:51.998990 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:51.999444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:51.999869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] PATH_INFO: `/v3/users/0dda7042b686482eb27324021cb8a811` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.001195 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dda7042b686482eb27324021cb8a811)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:52.003163 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:52.056013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dda7042b686482eb27324021cb8a811'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.065303 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dda7042b686482eb27324021cb8a811'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.065996 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dda7042b686482eb27324021cb8a811'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.066777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dda7042b686482eb27324021cb8a811'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.074872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dda7042b686482eb27324021cb8a811'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.095572 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1460/2892] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:27:51 2022] DELETE /identity/v3/users/0dda7042b686482eb27324021cb8a811 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:52.099947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.100745 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.101018 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.129013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:52.130276 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:52.164046 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=100 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:52.200821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1433/2893] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:52.210904 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:52.212360 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:52.288671 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237f055e-9034-4d42-bdfd-5f208a3968f7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:52.292456 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4473cac5-1fe9-441d-b2c4-9b5788fef2a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:52.293076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473cac5-1fe9-441d-b2c4-9b5788fef2a8 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.293348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473cac5-1fe9-441d-b2c4-9b5788fef2a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.293640 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4473cac5-1fe9-441d-b2c4-9b5788fef2a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.303632 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473cac5-1fe9-441d-b2c4-9b5788fef2a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:52.305092 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4473cac5-1fe9-441d-b2c4-9b5788fef2a8 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:52.315815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1461/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:27:52.644147 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:52.645888 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:52.659108 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5736fe97-2c5c-472e-8e2c-e6ee53cb9d81 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:52.664122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:52.664767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.665051 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.665533 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] PATH_INFO: `/v3/projects/4d5bef441a7b45af8f35acc2a91d5d3f` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.666695 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d5bef441a7b45af8f35acc2a91d5d3f)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:52.668612 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:52.734233 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d5bef441a7b45af8f35acc2a91d5d3f', 'request_id': 'req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.740818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d5bef441a7b45af8f35acc2a91d5d3f', 'request_id': 'req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.749563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d5bef441a7b45af8f35acc2a91d5d3f', 'request_id': 'req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.768288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] The token cache is being invalidate because project 4d5bef441a7b45af8f35acc2a91d5d3f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:27:52.768619 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.769235 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-3cf3c2b0-7052-48db-8749-9a0c4ed994f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:27:52.770767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1434/2895] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:27:52 2022] DELETE /identity/v3/projects/4d5bef441a7b45af8f35acc2a91d5d3f => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:27:52.775840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.776379 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.776934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.799455 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:52.800759 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:52.849538 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:52.874193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1462/2896] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:52.883300 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.883601 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.883860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.904280 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:52.905572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:52.914165 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1435/2897] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:52.918730 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:52.929811 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d926205-a7c4-44fe-bd55-66b1cc2d3392 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:52.934725 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:52.935443 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.935791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.936162 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.936921 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:27:52.937273 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:52.938968 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:52.952564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1463/2898] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:27:52.956646 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:52.977412 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a47137-fdbe-4126-b4c9-63de35e9da8d None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:52.983243 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:52.983928 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:52.984265 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:52.984600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:52.985312 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:52.987430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:53.067171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1436/2899] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:27:52 2022] POST /identity/v3/projects => generated 373 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:53.076380 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:53.092591 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf618340-34db-47dd-a839-b33ccf282dcf admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:53.178957 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:53.179486 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:53.179763 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:53.180031 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:53.180692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:53.182691 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:53.208693 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1464/2900] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:27:53 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:27:53.213286 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:53.225583 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72830cbc-3dd5-490d-958a-150d16c73333 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:53.230282 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:53.230881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:53.231159 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:53.232092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:53.232744 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:53.234393 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:53.245783 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1437/2901] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:53.249677 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:53.259978 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd773e-237c-4094-b00d-f5c7ae33dcd7 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:53.265998 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf334e-224b-4455-aa12-3cc9315c9513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:53.266564 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf334e-224b-4455-aa12-3cc9315c9513 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:53.266900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf334e-224b-4455-aa12-3cc9315c9513 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:53.267107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf334e-224b-4455-aa12-3cc9315c9513 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:53.267748 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf334e-224b-4455-aa12-3cc9315c9513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:53.269326 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf334e-224b-4455-aa12-3cc9315c9513 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:53.285361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1465/2902] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:27:53.286964 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:53.297824 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9566a1e-250e-4b8f-8a90-44ae3ada003a admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:53.303476 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:53.304375 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:53.304821 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:53.305250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] PATH_INFO: `/v3/projects/1ccb3a5bdf594a2da3c0bf92891081a4/users/7bff27bc79884b74966b1cb7f114a8ac/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:53.323900 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccb3a5bdf594a2da3c0bf92891081a4, user_id=7bff27bc79884b74966b1cb7f114a8ac, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:53.327150 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:53.342341 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1438/2903] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:27:53 2022] PUT /identity/v3/projects/1ccb3a5bdf594a2da3c0bf92891081a4/users/7bff27bc79884b74966b1cb7f114a8ac/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:27:53.352853 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706628f9-e352-4cb3-8271-d3711d3cafa5 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:53.353445 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706628f9-e352-4cb3-8271-d3711d3cafa5 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:53.354044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706628f9-e352-4cb3-8271-d3711d3cafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:53.387375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-706628f9-e352-4cb3-8271-d3711d3cafa5 None None] MFA Rules not processed for user `7bff27bc79884b74966b1cb7f114a8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:53.389143 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-706628f9-e352-4cb3-8271-d3711d3cafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:53.460404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1466/2904] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:27:53.471436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:27:53.473146 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:53.534378 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c422f7-8566-440a-a958-ab341d9aeb25 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:27:53.538040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:27:53.538649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:53.538927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:53.539244 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:53.548324 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:27:53.549768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:27:53.553563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:27:53.560658 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1439/2905] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:27:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:27:56.091365 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:27:56.091979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:27:56.091979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:27:56.113397 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] MFA Rules not processed for user `7bff27bc79884b74966b1cb7f114a8ac`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:27:56.114667 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:27:56.121588 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1467/2906] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:27:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:28:02.446204 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:02.447773 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:02.504401 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3937ec2-6195-47ca-9539-523f640146b0 service cinder] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:02.508551 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:02.509145 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:02.509433 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:02.509693 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:02.511473 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:02.522081 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:02.523501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:02.532740 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1440/2907] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:28:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:28:06.106770 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:28:07.658809 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:07.661250 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:07.740193 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c04acb6-ee45-417b-a7e2-ca38e901a2cd None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:07.746869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:07.747630 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:07.747913 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:07.748185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:07.750100 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:07.763080 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:07.764646 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5eab3c-22a3-4f18-a3e6-8c8b79ef0e22 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:07.775555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1468/2908] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:28:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:28:11.727588 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:11.741458 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fced1da-76d2-4a2b-a0a7-262ffe79f2b4 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:11.748449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:11.749296 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:11.750396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:11.750396 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:11.752199 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:11.768013 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:11.769763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:11.782087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1441/2909] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:28:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:28:17.770690 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=34 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:28:20.093790 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3ea4-a1e1-4f7a-9764-7547744cc658 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:20.094354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3ea4-a1e1-4f7a-9764-7547744cc658 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:20.094354 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8c3ea4-a1e1-4f7a-9764-7547744cc658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:20.114373 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ad8c3ea4-a1e1-4f7a-9764-7547744cc658 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:28:20.115577 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ad8c3ea4-a1e1-4f7a-9764-7547744cc658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:20.171799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1469/2910] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:28:20.183916 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:20.185277 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:20.251318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b5b604-11f5-4cd2-b421-6df188a59744 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:20.256260 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:20.256850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:20.257126 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:20.257392 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:20.265651 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:20.267228 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:20.275373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1442/2911] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:28:20.772744 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa67d6-991f-4f2c-9e2f-4c71148ce45e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:20.773024 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa67d6-991f-4f2c-9e2f-4c71148ce45e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:20.773295 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa67d6-991f-4f2c-9e2f-4c71148ce45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:20.792156 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-51fa67d6-991f-4f2c-9e2f-4c71148ce45e None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:28:20.793336 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-51fa67d6-991f-4f2c-9e2f-4c71148ce45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:20.800337 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1470/2912] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 20:28:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:28:20.813131 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:20.823595 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d128e6bd-5e0f-4b43-8523-51a6201e1770 service swift] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:20.829373 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:20.830123 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:20.830391 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:20.830657 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:20.838884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:20.840516 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:20.848415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1443/2913] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:28:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:28:21.081579 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fd7a-0c4b-413d-85be-0335cf227ed9 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:21.081896 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fd7a-0c4b-413d-85be-0335cf227ed9 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:21.082177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b2fd7a-0c4b-413d-85be-0335cf227ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:21.101053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-54b2fd7a-0c4b-413d-85be-0335cf227ed9 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:28:21.102277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-54b2fd7a-0c4b-413d-85be-0335cf227ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:21.109800 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1471/2914] 10.222.0.15 () {58 vars in 1009 bytes} [Tue Sep 13 20:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:28:21.122030 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:21.132615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d321b3-862e-44e6-8d79-55af52cd4402 service swift] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:21.137610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:21.138218 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:21.138496 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:21.138767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:21.146696 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:21.148238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:21.156351 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1444/2915] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:28:21.474552 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:21.474833 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:21.475106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:21.495209 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:28:21.496404 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:21.503419 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1472/2916] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 20:28:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:28:21.515471 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:21.525692 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34e5f1f-feef-42b2-8c46-5a45af74757b service swift] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:21.531111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3b6c87-3fe7-4e29-867a-72e869adc703 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:21.531685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6c87-3fe7-4e29-867a-72e869adc703 service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:21.531957 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6c87-3fe7-4e29-867a-72e869adc703 service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:21.532495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b6c87-3fe7-4e29-867a-72e869adc703 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:21.540262 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b6c87-3fe7-4e29-867a-72e869adc703 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:21.541835 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3b6c87-3fe7-4e29-867a-72e869adc703 service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:21.550246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1445/2917] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:28:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:28:23.609842 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:23.689570 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07f434-be56-457a-a681-f3f982cf6e4e None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:23.694003 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:23.694619 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:23.694902 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:23.695168 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] PATH_INFO: `/v3/users/7bff27bc79884b74966b1cb7f114a8ac` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:23.696369 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bff27bc79884b74966b1cb7f114a8ac)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:23.698297 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:23.748849 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bff27bc79884b74966b1cb7f114a8ac'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:23.756050 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bff27bc79884b74966b1cb7f114a8ac'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:23.756663 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bff27bc79884b74966b1cb7f114a8ac'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:23.757262 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bff27bc79884b74966b1cb7f114a8ac'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:23.764734 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bff27bc79884b74966b1cb7f114a8ac'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:23.769786 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:28:23.783635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1473/2918] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:28:23 2022] DELETE /identity/v3/users/7bff27bc79884b74966b1cb7f114a8ac => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:28:23.788206 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:23.788659 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:23.788747 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:23.794790 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:28:23.820310 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:28:23.821501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:23.881072 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1446/2919] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:28:23.891323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:23.892707 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:23.948051 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52e0d4c-5dcf-490c-9779-fb3f4570b6e9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:23.952189 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:23.952777 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:23.953049 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:23.953316 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:23.962097 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:23.963494 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:23.972843 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1474/2920] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:28:24.140818 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:24.142415 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:24.158059 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408144aa-7e2b-440b-9285-bd618ba4ab02 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:24.163565 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:24.164268 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:24.164453 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:24.164714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] PATH_INFO: `/v3/projects/1ccb3a5bdf594a2da3c0bf92891081a4` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:24.165899 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccb3a5bdf594a2da3c0bf92891081a4)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:24.168436 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:24.218482 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccb3a5bdf594a2da3c0bf92891081a4', 'request_id': 'req-08d31a90-3bca-4da6-9578-d21da673598b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:24.224816 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccb3a5bdf594a2da3c0bf92891081a4', 'request_id': 'req-08d31a90-3bca-4da6-9578-d21da673598b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:24.233318 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccb3a5bdf594a2da3c0bf92891081a4', 'request_id': 'req-08d31a90-3bca-4da6-9578-d21da673598b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:24.244752 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] The token cache is being invalidate because project 1ccb3a5bdf594a2da3c0bf92891081a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:28:24.245040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:24.246363 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:28:24.247860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1447/2921] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:28:24 2022] DELETE /identity/v3/projects/1ccb3a5bdf594a2da3c0bf92891081a4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:28:25.610765 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:28:25.619344 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670dfe1a-4ad0-4b4c-a80f-b438c0cfd70b service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:28:25.666555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:28:25.667145 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:28:25.667920 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:28:25.668185 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:28:25.670079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:28:25.730738 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:28:25.732146 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8d6c4-1bfb-4fbf-8179-11fc72d8dd35 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:28:25.741680 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1475/2922] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:28:25 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:28:34.241712 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:28:35.737368 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=81,keystone:DELETE=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:30:01.294975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:30:01.296395 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:30:01.422685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d31a90-3bca-4da6-9578-d21da673598b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:30:01.427885 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:30:01.428486 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:30:01.428768 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:30:01.429038 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:30:01.430774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:30:01.442065 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:30:01.443614 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:30:01.453609 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1448/2923] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:30:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:30:11.449225 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=58 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:30:53.946251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:30:53.946768 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:30:53.946865 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:30:53.966454 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] MFA Rules not processed for user `fce06803dd124d6680a9ce5792e7ea90`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:30:53.967645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:30:54.021200 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1476/2924] 10.222.0.15 () {58 vars in 967 bytes} [Tue Sep 13 20:30:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:30:54.065896 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:30:54.067288 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:30:54.135142 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1e18e-700c-45a7-82f2-c202568b4c4d service placement] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:30:54.139903 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '38d08c9aa2604871bec462424708b85b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:30:54.140495 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:30:54.140765 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:30:54.141033 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:30:54.149330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:30:54.150915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:30:54.159052 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1449/2925] 10.222.0.15 () {60 vars in 1384 bytes} [Tue Sep 13 20:30:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 20:31:00.894263 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:00.895788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:00.963371 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e76bf-ada5-4377-9a10-44304b95df55 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:00.967841 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:00.968460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:00.968741 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:00.969007 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] PATH_INFO: `/v3/users/03d9fa985d7343528e60455e0d00da65` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:00.970167 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=03d9fa985d7343528e60455e0d00da65)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:00.972062 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:01.016872 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03d9fa985d7343528e60455e0d00da65'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.024308 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03d9fa985d7343528e60455e0d00da65'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.024927 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03d9fa985d7343528e60455e0d00da65'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.025556 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03d9fa985d7343528e60455e0d00da65'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.032824 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03d9fa985d7343528e60455e0d00da65'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.050110 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1477/2926] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:00 2022] DELETE /identity/v3/users/03d9fa985d7343528e60455e0d00da65 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:01.221965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:01.234195 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47055400-42bf-4c78-ba51-dc906c25bd97 service swift] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:01.290688 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:01.291246 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:01.291525 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:01.291800 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] PATH_INFO: `/v3/projects/cd434e2ee3bd466abe5df77da23961f8` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:01.292927 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd434e2ee3bd466abe5df77da23961f8)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:01.294857 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:01.339548 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd434e2ee3bd466abe5df77da23961f8', 'request_id': 'req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.345398 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd434e2ee3bd466abe5df77da23961f8', 'request_id': 'req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.352780 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd434e2ee3bd466abe5df77da23961f8', 'request_id': 'req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.363010 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] The token cache is being invalidate because project cd434e2ee3bd466abe5df77da23961f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:01.363295 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.363902 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.365884 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1450/2927] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:01 2022] DELETE /identity/v3/projects/cd434e2ee3bd466abe5df77da23961f8 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:01.370107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:01.371540 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:01.385305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a61ec8-f19a-4ec3-876d-a532333c91ad admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:01.404689 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:01.442327 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:01.442953 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:01.443234 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:01.444107 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] PATH_INFO: `/v3/users/6d86d3d135bd48ed83064e675a48f5e8` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:01.445239 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d86d3d135bd48ed83064e675a48f5e8)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:01.447187 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:01.491791 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d86d3d135bd48ed83064e675a48f5e8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.499510 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d86d3d135bd48ed83064e675a48f5e8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.500119 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d86d3d135bd48ed83064e675a48f5e8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.500721 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d86d3d135bd48ed83064e675a48f5e8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.508192 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-a45f6a6c-ada3-4f91-9983-87de8e4232ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d86d3d135bd48ed83064e675a48f5e8'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.524973 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1478/2928] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:31:01 2022] DELETE /identity/v3/users/6d86d3d135bd48ed83064e675a48f5e8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:01.695585 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:01.697097 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:01.713118 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed68acd7-a631-47ac-b7fc-e5267ea4e92b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:01.731111 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:01.775798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:01.776427 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:01.776705 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:01.777006 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] PATH_INFO: `/v3/projects/d6bc4c61c4b549cf99d6bae637ba5641` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:01.778155 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6bc4c61c4b549cf99d6bae637ba5641)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:01.780057 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:01.825949 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6bc4c61c4b549cf99d6bae637ba5641', 'request_id': 'req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.831917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6bc4c61c4b549cf99d6bae637ba5641', 'request_id': 'req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.841789 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6bc4c61c4b549cf99d6bae637ba5641', 'request_id': 'req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.852234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] The token cache is being invalidate because project d6bc4c61c4b549cf99d6bae637ba5641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93113) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:31:01.852524 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.853136 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-932ac4ae-10b5-4f0c-a43d-0c49acaf071f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:31:01.854649 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1451/2929] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:31:01 2022] DELETE /identity/v3/projects/d6bc4c61c4b549cf99d6bae637ba5641 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:01.860043 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:01.860357 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:01.860634 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:01.880757 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:01.882484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:01.940593 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1479/2930] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:01.948915 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:01.949197 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:01.949472 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:01.968925 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:01.970160 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:01.977971 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1452/2931] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:01.982001 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:01.991984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26719d00-61e4-4966-bcd5-27c0b91b4d75 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:01.996565 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:01.997160 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:01.997440 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:01.997714 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] PATH_INFO: `/v3/domains` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:01.998387 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=93112) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 20:31:01.998692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.000268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.012076 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1480/2932] 10.222.0.15 () {58 vars in 1153 bytes} [Tue Sep 13 20:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 20:31:02.016205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.025275 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59980007-3e8a-4054-94a6-d66c91cf6e7a None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.029983 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.030812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.031082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.031348 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.031970 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.033850 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.060092 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1453/2933] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:02 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:02.064106 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.075339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88a47-f328-4641-b0a4-aa200afec004 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.113799 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:02.143939 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.144544 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.144816 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.145082 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.145705 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.147681 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.167782 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1481/2934] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:02 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:02.171917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.182647 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a852cd1-6a3b-44a3-98ad-2ffa612a2d15 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.187210 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.187798 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.188074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.188344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.188979 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.190602 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.203035 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1454/2935] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:02.207039 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.216793 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf57a95-fd87-4aac-ae4a-62af299a9da4 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.221555 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df5dad-f51f-4e4a-a45e-8e54e26b7e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.222191 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df5dad-f51f-4e4a-a45e-8e54e26b7e73 admin admin] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.222467 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df5dad-f51f-4e4a-a45e-8e54e26b7e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.222735 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df5dad-f51f-4e4a-a45e-8e54e26b7e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.223372 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df5dad-f51f-4e4a-a45e-8e54e26b7e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.225044 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df5dad-f51f-4e4a-a45e-8e54e26b7e73 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.235567 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1482/2936] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:02.239479 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.249893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd42163b-dd90-4580-9905-859a096ca45b admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.254540 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.255171 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] REQUEST_METHOD: `PUT` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.255448 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.255763 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] PATH_INFO: `/v3/projects/c95ce3183d004ddc82056e7a95e02920/users/29b9ce470c0d4ab7a4cbefb1e213b972/roles/61f630f0ba254ac5bb25aa078b9876a0` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.271413 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c95ce3183d004ddc82056e7a95e02920, user_id=29b9ce470c0d4ab7a4cbefb1e213b972, role_id=61f630f0ba254ac5bb25aa078b9876a0)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.274047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.288073 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1455/2937] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:02 2022] PUT /identity/v3/projects/c95ce3183d004ddc82056e7a95e02920/users/29b9ce470c0d4ab7a4cbefb1e213b972/roles/61f630f0ba254ac5bb25aa078b9876a0 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:02.292952 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b19b7b-6f22-42e2-94f5-ac2a395a1b61 None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.293225 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b19b7b-6f22-42e2-94f5-ac2a395a1b61 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.293503 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b19b7b-6f22-42e2-94f5-ac2a395a1b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.312755 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-66b19b7b-6f22-42e2-94f5-ac2a395a1b61 None None] MFA Rules not processed for user `8e9932faac4043979f160bcfef7916ea`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:02.313993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-66b19b7b-6f22-42e2-94f5-ac2a395a1b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:02.370420 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1483/2938] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 20:31:02.379876 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:02.382106 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:02.434960 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:02.441330 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20835f43-75bc-40f2-a4a4-5daaf144ae1f admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:02.446163 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:02.446947 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:02.447230 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:02.447505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:02.456505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:02.457975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:02.467775 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1456/2939] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:06.063255 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.063819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.063819 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.092275 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] MFA Rules not processed for user `29b9ce470c0d4ab7a4cbefb1e213b972`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:06.093509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:06.160240 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1484/2940] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:31:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:06.165344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.174863 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abebef9d-4aa2-4dc7-9799-921ef3aec26b service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.179615 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.180205 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.180477 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.180742 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] PATH_INFO: `/v3/projects` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.181369 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.183273 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.212383 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1457/2941] 10.222.0.15 () {60 vars in 1153 bytes} [Tue Sep 13 20:31:06 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 20:31:06.216401 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.228960 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1772d3a-2329-47f7-891e-dd0fa7dc9c8b None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.267252 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:06.288606 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.289213 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.290013 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.290346 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] PATH_INFO: `/v3/users` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.290906 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.292814 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.328277 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1485/2942] 10.222.0.15 () {60 vars in 1144 bytes} [Tue Sep 13 20:31:06 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 20:31:06.331703 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.343087 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899a7b7-e3a4-4538-949c-88e795ab3587 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.350404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.351194 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.351628 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.351898 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.352545 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.354162 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.391812 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1458/2943] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:06.396175 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.407305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74fd8c-5e8e-4b00-a2e5-76e8cc83a145 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.415988 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.416937 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.417219 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.417491 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] PATH_INFO: `/v3/projects/7b118d8a877340648111f8b89ebccaa1/users/e9f5c14804b64f82ac7790c35f8417cd/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.440017 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b118d8a877340648111f8b89ebccaa1, user_id=e9f5c14804b64f82ac7790c35f8417cd, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.442958 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.461938 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1486/2944] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:06 2022] PUT /identity/v3/projects/7b118d8a877340648111f8b89ebccaa1/users/e9f5c14804b64f82ac7790c35f8417cd/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:31:06.464041 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.475598 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2f5bd0-603c-4390-b9ed-eed454320fc7 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.543611 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4825a9-70ff-4098-8020-19de23c4b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.544234 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4825a9-70ff-4098-8020-19de23c4b8f0 admin admin] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.544505 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4825a9-70ff-4098-8020-19de23c4b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.544772 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4825a9-70ff-4098-8020-19de23c4b8f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.545421 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4825a9-70ff-4098-8020-19de23c4b8f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.547116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4825a9-70ff-4098-8020-19de23c4b8f0 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.557774 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1459/2945] 10.222.0.15 () {58 vars in 1122 bytes} [Tue Sep 13 20:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 20:31:06.561900 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:06.573325 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b8349-106f-4f46-8c69-a1047d3be018 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:06.577974 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:06.578635 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:06.578910 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:06.579180 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] PATH_INFO: `/v3/projects/7b118d8a877340648111f8b89ebccaa1/users/e9f5c14804b64f82ac7790c35f8417cd/roles/40a9647e8ec745c999c2457d053557a7` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:06.581122 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b118d8a877340648111f8b89ebccaa1, user_id=e9f5c14804b64f82ac7790c35f8417cd, role_id=40a9647e8ec745c999c2457d053557a7)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:06.584746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:06.595950 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1487/2946] 10.222.0.15 () {60 vars in 1483 bytes} [Tue Sep 13 20:31:06 2022] PUT /identity/v3/projects/7b118d8a877340648111f8b89ebccaa1/users/e9f5c14804b64f82ac7790c35f8417cd/roles/40a9647e8ec745c999c2457d053557a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 20:31:10.910489 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86cca9-06eb-4235-9995-54a4918d6045 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:10.910939 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86cca9-06eb-4235-9995-54a4918d6045 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:10.911054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86cca9-06eb-4235-9995-54a4918d6045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:10.941640 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-4d86cca9-06eb-4235-9995-54a4918d6045 None None] MFA Rules not processed for user `e9f5c14804b64f82ac7790c35f8417cd`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:10.943576 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-4d86cca9-06eb-4235-9995-54a4918d6045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:11.040130 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1460/2947] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:31:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:11.056412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:11.058692 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:11.153685 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674336a-3a6a-4b75-9a43-6bb097da3a32 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:11.159815 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:11.160441 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:11.160788 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:11.161068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:11.223916 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:11.226293 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:11.236133 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1488/2948] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:31:12.204864 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:12.205335 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:12.205449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:12.224879 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] MFA Rules not processed for user `29b9ce470c0d4ab7a4cbefb1e213b972`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:31:12.226736 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:12.233521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1461/2949] 10.222.0.15 () {58 vars in 958 bytes} [Tue Sep 13 20:31:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:31:21.231808 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:22.218767 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=85,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:35.935485 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:35.936867 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:36.002881 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603bafc9-e116-4f1c-8f35-a89e2934b4bd service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:36.006458 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f8ea3cf81f944a2b02e59b3ff049f6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:36.007054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:36.007334 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:36.008116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:36.009859 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:36.022840 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:36.024880 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:36.035993 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1489/2950] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:37.017761 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:37.078382 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744b259-6dee-4acc-9aef-cd767ff3ed24 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:37.082078 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:37.082671 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:37.082946 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:37.083212 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:37.085029 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:37.099632 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:37.101054 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:37.112829 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1462/2951] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:46.031111 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=31 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:47.107702 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=26 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:31:53.702305 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:53.711826 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf1a65-d433-492a-83d3-2c352f81140b service cinder] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:53.715461 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:53.716086 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:53.716359 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:53.716654 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:53.730626 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:53.732068 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:53.743306 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1490/2952] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:31:55.286329 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:31:55.299880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b58a03-8cbb-4aee-8cd9-a3ec517685f0 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:31:55.307385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:31:55.308122 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:31:55.308404 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:31:55.308672 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:31:55.310501 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:31:55.324021 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:31:55.325764 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:31:55.352426 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1463/2953] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:31:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:32:03.738444 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:32:05.332741 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=6 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:33:02.236000 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:33:02.237447 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:02.267518 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858bb74-6af3-4f96-9fd8-4b1e75fd8b8b service neutron] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:33:02.271158 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:33:02.271750 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:33:02.272028 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:33:02.272300 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:33:02.274614 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:33:02.284108 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:33:02.285520 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fec939-37e9-4b96-8409-b109b82d5759 service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:33:02.293309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1491/2954] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:33:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:33:12.289797 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=15 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:25.469715 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:25.488518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58c8e5-a65a-4467-a264-ce9a7ec7bfcf service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:25.492109 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:25.492685 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.492956 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.493221 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.494889 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:25.503975 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:25.505357 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc35bbc-6e53-4d89-8a17-9c6fb1a5b78e service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:25.513655 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1464/2955] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:25.633232 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5e52-4a05-4487-8b8f-9b91117f4954 None None] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.633831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5e52-4a05-4487-8b8f-9b91117f4954 None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.633831 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5e52-4a05-4487-8b8f-9b91117f4954 None None] PATH_INFO: `/` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.635140 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1492/2956] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 20:34:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 20:34:25.641250 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ada7f-bd26-4cc1-a6cb-ddf9c797f4e1 None None] REQUEST_METHOD: `POST` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.641539 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ada7f-bd26-4cc1-a6cb-ddf9c797f4e1 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.641832 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583ada7f-bd26-4cc1-a6cb-ddf9c797f4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.681385 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.auth.core [None req-583ada7f-bd26-4cc1-a6cb-ddf9c797f4e1 None None] MFA Rules not processed for user `36fab0873915497cada62e8ea73bd4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=93113) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:25.682531 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-583ada7f-bd26-4cc1-a6cb-ddf9c797f4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:25.689569 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1465/2957] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 20:34:25 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:25.702934 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd5e52-4a05-4487-8b8f-9b91117f4954 None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:25.713611 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd5e52-4a05-4487-8b8f-9b91117f4954 None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:25.717348 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239b2c4-d0cf-48fc-a97f-41e56904088b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:25.717971 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b2c4-d0cf-48fc-a97f-41e56904088b service neutron] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.718251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b2c4-d0cf-48fc-a97f-41e56904088b service neutron] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.718523 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239b2c4-d0cf-48fc-a97f-41e56904088b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.725537 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b2c4-d0cf-48fc-a97f-41e56904088b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:25.726979 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239b2c4-d0cf-48fc-a97f-41e56904088b service neutron] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:25.734672 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1493/2958] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:25.817590 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd1b34-0ea5-4a82-99a6-8e490846b0b8 None None] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.817909 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd1b34-0ea5-4a82-99a6-8e490846b0b8 None None] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.818184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dd1b34-0ea5-4a82-99a6-8e490846b0b8 None None] PATH_INFO: `/` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.819572 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1466/2959] 10.222.0.15 () {54 vars in 921 bytes} [Tue Sep 13 20:34:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 20:34:25.824751 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] REQUEST_METHOD: `POST` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.825035 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.825311 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.877405 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.auth.core [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] MFA Rules not processed for user `910ca69bc9bf4f328d9537f0b9bb1658`. Rule list: `[]` (Enabled: `True`). {{(pid=93112) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 20:34:25.878638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:25.903897 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1494/2960] 10.222.0.15 () {58 vars in 1018 bytes} [Tue Sep 13 20:34:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 20:34:25.914873 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd1b34-0ea5-4a82-99a6-8e490846b0b8 None None] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:25.925710 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dd1b34-0ea5-4a82-99a6-8e490846b0b8 None None] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:25.929182 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:25.929779 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:25.930061 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:25.930328 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:25.940067 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:25.941400 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:25.951852 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1467/2961] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:26.012798 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:26.014196 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:26.081351 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b1f31-93ec-4e3d-8e43-3869d4acb0be None None] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:26.087014 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:26.087608 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:26.087885 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:26.088153 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:26.089877 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:26.100948 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:26.102551 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:26.112211 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1495/2962] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:27.428817 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:34:27.440563 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab9b9c-141e-4cf1-a9ca-95e5dc470d09 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:34:27.446028 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:34:27.446645 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:34:27.446917 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:34:27.447184 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:34:27.448860 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:34:27.461865 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:34:27.463625 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:34:27.476172 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1468/2963] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:34:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:34:36.108052 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=59 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:34:37.470403 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=32 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:35:06.291631 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:35:06.302061 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934dd008-f9c2-4c28-b88b-1e87317b4364 service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:35:06.307484 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:35:06.308072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:35:06.308375 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:35:06.308645 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:35:06.310361 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:35:06.322227 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:35:06.323779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:35:06.333560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1496/2964] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:35:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:35:16.329309 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=6 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:36:09.980173 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:09.992515 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3011ea6-2086-4a4d-b02b-f3990e630790 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:09.998626 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:09.999294 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:09.999661 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:10.000002 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:10.002040 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:10.017920 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:10.019610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:10.031633 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1469/2965] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:36:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:36:11.284514 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:11.286009 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:11.304779 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8894a-53c6-4f22-bc35-76fcc31502a2 service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:11.310928 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '910ca69bc9bf4f328d9537f0b9bb1658', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:11.311531 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:11.311810 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:11.312079 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:11.322056 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:11.323620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:11.332618 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1497/2966] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:36:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:36:13.838739 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:13.848666 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089b3cb-b476-46b3-ac22-cf9e5a9e5813 service nova] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:13.852283 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36fab0873915497cada62e8ea73bd4d6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:13.852880 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:13.853156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:13.853430 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:13.914060 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:13.915399 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:13.925521 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1470/2967] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:36:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:36:16.969869 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:16.974132 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:16.995620 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eec3362-5474-45eb-8742-6944960ac701 service nova] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:17.003746 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:17.004560 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] REQUEST_METHOD: `GET` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:17.005016 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:17.005460 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:17.008733 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:17.031205 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:17.032975 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:17.044884 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1498/2968] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:36:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 20:36:20.169953 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:20.180424 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c837bcb-1842-4542-8dca-08e3089f3751 service neutron] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:20.186340 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:20.186967 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:20.187238 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:20.187513 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] PATH_INFO: `/v3/users/29b9ce470c0d4ab7a4cbefb1e213b972` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:20.188630 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b9ce470c0d4ab7a4cbefb1e213b972)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:20.190600 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:20.236610 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b9ce470c0d4ab7a4cbefb1e213b972'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.243507 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b9ce470c0d4ab7a4cbefb1e213b972'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.244727 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b9ce470c0d4ab7a4cbefb1e213b972'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.245344 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b9ce470c0d4ab7a4cbefb1e213b972'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.252315 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b9ce470c0d4ab7a4cbefb1e213b972'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.269965 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1471/2969] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:36:20 2022] DELETE /identity/v3/users/29b9ce470c0d4ab7a4cbefb1e213b972 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:36:20.439274 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:20.450268 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f0a25-17f4-4c5d-a444-6fc7f9f03670 service placement] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:20.509251 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:20.510553 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:20.510967 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:20.511246 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] PATH_INFO: `/v3/projects/c95ce3183d004ddc82056e7a95e02920` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:20.512417 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c95ce3183d004ddc82056e7a95e02920)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:20.514412 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:20.562638 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c95ce3183d004ddc82056e7a95e02920', 'request_id': 'req-e51ad812-1441-4534-a4a8-3eb9959391f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.569036 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c95ce3183d004ddc82056e7a95e02920', 'request_id': 'req-e51ad812-1441-4534-a4a8-3eb9959391f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.577087 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c95ce3183d004ddc82056e7a95e02920', 'request_id': 'req-e51ad812-1441-4534-a4a8-3eb9959391f9'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.588054 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] The token cache is being invalidate because project c95ce3183d004ddc82056e7a95e02920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:36:20.588339 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.589072 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.590775 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1499/2970] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:36:20 2022] DELETE /identity/v3/projects/c95ce3183d004ddc82056e7a95e02920 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:36:20.594714 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:20.596332 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:20.610241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14416ad-274d-44f9-bd29-5cad30999500 admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:20.657836 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:36:20.667825 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:20.668449 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:20.668726 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:20.668997 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] PATH_INFO: `/v3/users/e9f5c14804b64f82ac7790c35f8417cd` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:20.670149 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f5c14804b64f82ac7790c35f8417cd)` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:20.672047 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:20.717988 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f5c14804b64f82ac7790c35f8417cd'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.724881 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f5c14804b64f82ac7790c35f8417cd'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.725518 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f5c14804b64f82ac7790c35f8417cd'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.726137 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f5c14804b64f82ac7790c35f8417cd'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.733478 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.notifications [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f5c14804b64f82ac7790c35f8417cd'} {{(pid=93113) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:20.750508 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1472/2971] 10.222.0.15 () {58 vars in 1224 bytes} [Tue Sep 13 20:36:20 2022] DELETE /identity/v3/users/e9f5c14804b64f82ac7790c35f8417cd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:36:20.919471 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Authenticating user token {{(pid=93112) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:36:20.921011 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.fernet_utils [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93112) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:36:20.935682 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ad812-1441-4534-a4a8-3eb9959391f9 admin admin] Validating token access rules against request {{(pid=93112) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:36:20.992878 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8e9932faac4043979f160bcfef7916ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '35cfa2e060644668a62b0b52ed4b9871', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93112) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:36:20.993512 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:36:20.993829 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] SCRIPT_NAME: `/identity` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:36:20.994102 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] PATH_INFO: `/v3/projects/7b118d8a877340648111f8b89ebccaa1` {{(pid=93112) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:36:20.995212 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b118d8a877340648111f8b89ebccaa1)` {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:36:20.997116 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] RBAC: Authorization granted {{(pid=93112) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:36:21.003323 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:36:21.045529 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b118d8a877340648111f8b89ebccaa1', 'request_id': 'req-f8c7abfe-7c3e-4efa-802c-52e086ace958'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:21.051984 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b118d8a877340648111f8b89ebccaa1', 'request_id': 'req-f8c7abfe-7c3e-4efa-802c-52e086ace958'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:21.058818 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b118d8a877340648111f8b89ebccaa1', 'request_id': 'req-f8c7abfe-7c3e-4efa-802c-52e086ace958'} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:21.069129 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] The token cache is being invalidate because project 7b118d8a877340648111f8b89ebccaa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=93112) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 20:36:21.069416 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:21.070053 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG keystone.notifications [None req-f8c7abfe-7c3e-4efa-802c-52e086ace958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=93112) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 20:36:21.071509 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: [pid: 93112|app: 0|req: 1500/2972] 10.222.0.15 () {58 vars in 1233 bytes} [Tue Sep 13 20:36:20 2022] DELETE /identity/v3/projects/7b118d8a877340648111f8b89ebccaa1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 20:36:30.747211 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=29,keystone:DELETE=10,keystone:INSERT=1 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:36:31.066177 dv-cinder-828807-6-iscsi devstack@keystone.service[93112]: DEBUG dbcounter [-] [93112] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=93112) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 20:40:08.212641 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Authenticating user token {{(pid=93113) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 20:40:08.214074 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:08.343893 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4ccf4d-d42c-4709-9cef-f11ca5b1fecf admin admin] Validating token access rules against request {{(pid=93113) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 20:40:08.402241 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f3be656bf6446f99a23910cd833d799', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ffb76500ee954275b63e4afa84c5b184', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=93113) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 20:40:08.402839 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] REQUEST_METHOD: `GET` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 20:40:08.403116 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] SCRIPT_NAME: `/identity` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 20:40:08.403389 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=93113) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 20:40:08.405082 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.fernet_utils [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=93113) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 20:40:08.416156 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 20:40:08.417691 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaccea0e-a922-4efe-ae89-15d5c626bcea service placement] RBAC: Authorization granted {{(pid=93113) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 20:40:08.427480 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: [pid: 93113|app: 0|req: 1473/2973] 10.222.0.15 () {60 vars in 1365 bytes} [Tue Sep 13 20:40:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 20:40:18.423200 dv-cinder-828807-6-iscsi devstack@keystone.service[93113]: DEBUG dbcounter [-] [93113] Writing DB stats keystone:SELECT=84 {{(pid=93113) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}